Intro to Epic: Tame the iSWAP

[CamilloMoschner]

Hi everyone,

This is a big one:

The internal Swivel Arm Plate Handler (iSWAP) is one of Hamilton STAR(let)'s most powerful device Capabilities.

It enables…

• movement of resources with force-sensed feedback for secure pickup, transfer and placement
• movement to off-deck sites due to its swivel/SCARA design → ~90 mm out of the main chassis to the left, and ~20 mm out to the right
• instantly enables horizontal integration of other devices into a workcell, i.e. without the need for additional arms

Depending on the model and firmware version it can pick up plates in either just portrait mode or portrait and landscape mode.

However, even in the OEM software the iSWAP is infamous for being a bit difficult to use.

Since we have total control over the devices we own with PyLabRobot and like to make the difficult parts of our automation life easy, I started an epic (long-term project with multiple sub-components) to make the iSWAP smarter and easier to use.

I call this epic “Tame the iSWAP”.

In this thread I would like to showcase some of the current issues with the iSWAP and how we are openly developing it into a more advanced resource transfer system:

Background: iSWAP Geometry

Fundamentally the iSWAP has 3 components:

1. A Cartesian robot - moves the base/rotation drive/θ₁ of the iSWAP in…
   • x` (via the X-arm motor),
   • `y` (via the iSWAP channel y-motor), and
   • `z` (via the iSWAP channel z-motor).
2. A SCARA robot
   • 2 joints - representing…
     • the “rotation drive” θ₁
     • the “wrist drive” θ₂
       and
   • 2 linkages
     • linkage_1: rotation drive ↔ wrist drive
     • linkage_2: wrist drive ↔ gripper center
3. A Gripper with a linear motor

**More details on the SCARA parts**:

Each joint has its own specific range of motion and associated with their specific values an Orientation *with regard to* (`wrt`) the STAR coordinate system:

• Joint 1: θ₁ has 3 “standard” yaw angles (Orientation around the Z-axis)

  • -90° / ~ -29_000 increments → LEFT (`wrt` the STAR)
  • 0° / ~ 0 increments → FRONT
  • 90° / ~ 29_000 increments → RIGHT

• Joint 2: θ₂ has 4 “standard” yaw angles (Orientation around the Z-axis)

  • -135° / ≈ -26_577 increments → RIGHT
  • -45° / ≈ -8_860 increments → STRAIGHT
  • +45° / ≈ +9_044 increments → LEFT
  • +135° / ≈ +26_858 increments → REVERSE

This means there are 3 x 4 = 12 “standard” iSWAP-SCARA poses.

Moving from any given pose to any other therefore gives 12 x 11 = 132 nontrivial standard movement patterns.

Note: there is no reason why the hardware-firmware would not be able to move to any “non-standard” pose, i.e. arbitrary joint configurations (e.g. θ₁=-20°,θ₂=100°).

Current Issues

What is the problem?

The standard commands we currently have in PLR to use the iSWAP are based on the `C0` command module.

It is too (inconsistently) smart

When using our current `.iswap_get_plate()` / `C0PP` or `.move_resource()` / `C0PM` commands we only tell the commands the “grip direction” the gripper should have with regard to (`wrt`) the STAR’s entire coordinate system:

i.e. when saying `GripperDirection=Back` the gripper fingers point to the front, and the gripper is positioned behind the resource being picked up.

But … what is the angle of the two joints?

With `GripperDirection=Back` there are 3 different joint configurations which can achieve this GripDirection:

• θ₁=LEFT, θ₂=LEFT
• θ₁=FRONT, θ₂=STRAIGHT
• θ₁=RIGHT, θ₂=RIGHT

Which one is going to be chosen? … we don’t know - because the firmware makes this decision (with these ‘smart’ commands).

But the firmware has no deck awareness which means it might decide to orient the joints in a manner that crashes into another resource (e.g. a tip carrier next to our plate)… and we have no knowledge of when and why it would happen.

i.e. the currently exposed iSWAP movement commands are a black-box decision maker.

(Note: some movements are only possible in a specific joint configuration. e.g.: moving out of the main chassis to either side requires the linkages to straighten completely in that direction.

As a result, and confusingly, some commands do behave as we expect, clouding expected behaviour with position-dependency)

Firmware doesn’t cover all safety/risk scenarios

PLR already exposed some low-level commands to control some motors of the iSWAP assembly:

e.g. we can rotate joint_1 and joint_2 independently.

However, these commands do not perform a check whether…

1. all channels are out of the way (i.e. are they out of x-y reach for the upcoming iSWAP movement → side-crash risk), nor
2. out of z-height (i.e. whether the channels are raised so the iSWAP can move below → side-crash risk), nor
3. the channels are out of z-height but have tips mounted which bring the tips into the path of the iSWAP movement → channel stop-disk beheading risk

That is dangerous.

Potential Firmware Issues

It appears that everyone I know uses an iSWAP control board from 2012 - not an issue by itself (and the iSWAP is being replaced by the Internal Plate Gripper (IPG) in current STAR releases) but iSWAP-equipped STARs will be in service for years and I found some issues at the firmware level:

As part of the “Tame the iSWAP” project, I looked carefully into the joint angles of each movement.

When parked/just initialised → the iSWAP grip center position returned by the firmware doesn’t match the actual grip center:

This shows that the parking position overwrites the firmware return value for the iSWAP grip center to the center_x-center_y position of the joint_1 / rotation drive.

-> That is off by 32.8 mm in x and 138 mm in y

Interestingly, iSWAP movement commands in `x`, `y`, or combined joint rotation do *not* update the iSWAP grip center either!

This looks like it could be a firmware bug, though I can’t rule out that I’m missing context.

It indicates a crucial find:

`STARBackend.request_iswap_position()` / `C0QG` does *not* appear to perform a forward kinematics calculation of where the grip center is based on queried motor encoder state.

It seems likely to me that only certain firmware commands actually update an internal tracker of the state of the iSWAP instead.

For any form of more complex/advanced iSWAP behaviour we must have a command that reliably returns the actual grip center, no matter what commands have been executed beforehand.

Why do I say that only certain commands appear to update the firmware-internal tracker system:

From the lowest-level commands, only single angle rotations instantly update the tracker to the correct position.

i.e. the firmware commands inconsistently update the internal tracker.

Should we inform Hamilton directly? Considerations:

• OEMs don’t like know-it-alls
• Lab automation OEMs do not provide an incentive to do so (no bug bounties in this industry)
• Maybe we are wrong / don’t know the full picture, maybe they are aware and don’t consider it obstructive to most users’ needs
• It appears this bug has been around for 14 years … it might be that nobody has detected it because the thoroughness of PyLabRobot has not existed for most of that time period, or if detected a cost/benefit analysis might have concluded not to update but instead to build a new “Internal Plate Gripper” instead

…these are all speculations; please let me know what you think is happening here

What is happening?

Via a series of PRs I’ve created a plan to engineer us out of this dilemma:

1. Expose all position request commands, for all reference points.
2. Replace (presumably) broken firmware `park_iswap` with PLR-based transparent primitive and update `setup_iswap` with it.
3. Build forward kinematics model to always calculate the true position of the grip center.
4. Add advanced safety features
5. Build new commands from primitives which have well-defined default joint behaviour (e.g. always right-back) where possible, but log a warning if user hasn’t specified 2 intuitive parameters: joint_1 orientation AND grip direction (*both* now `wrt` STAR deck!).
6. Build (maybe simplified) inverse kinematics model that gives granular control over grip center speed and acceleration (with caveats accepted where the firmware does not expose the required granularity) - i.e. no more “violent spinning” + speed control for anti-splashing feature during transfer.

The current list of PRs in this epic:

• Part 1: Expose `iSWAPBackend.rotation_drive_request_position()` by BioCam · Pull Request #1011 · PyLabRobot/pylabrobot · GitHub

• Part 2: Expose `STARBackend.iswap_gripper_request_width()` by BioCam · Pull Request #1013 · PyLabRobot/pylabrobot · GitHub

• Part 3: Expose `STARBackend.iswap_rotation_drive_request_position()` by BioCam · Pull Request #1014 · PyLabRobot/pylabrobot · GitHub

• Part 4: Expose iSWAP per-drive predefined-position readers + rotation drive axis accessors by BioCam · Pull Request #1015 · PyLabRobot/pylabrobot · GitHub

• Part 5: Expose `STARBackend._iswap_rotate_both_joints_in_increments` for parallelised, smooth motion by BioCam · Pull Request #1016 · PyLabRobot/pylabrobot · GitHub

• Part 6: Expose X-axis movement with acceleration control for X-arm and iSWAP rotation drive by BioCam · Pull Request #1018 · PyLabRobot/pylabrobot · GitHub

• Part 7: Expose `STARBackend.iswap_rotation_drive_move_y()` by BioCam · Pull Request #1020 · PyLabRobot/pylabrobot · GitHub

• Part 8: Expose iswap rotation drive move z by rickwierenga · Pull Request #1021 · PyLabRobot/pylabrobot · GitHub (authored by Rick)

• Part 9: Expose iSWAP predefined Y/Z queries and drive-conversion helpers by BioCam · Pull Request #1023 · PyLabRobot/pylabrobot · GitHub

• Part 10: `STARBackend`: anchor iSWAP move_y to EEPROM parking pose by BioCam · Pull Request #1024 · PyLabRobot/pylabrobot · GitHub

This means we have just reached step 3 and 4.

What’s next?

We continue to move on to the next steps.

We wanted to inform the community about this big project and the new stories this will enable, e.g.:

• “iSWAP never crashes into the STAR anymore”
• “liquid doesn’t splash out of plate when moved via iSWAP”

…

future stories this enables us to build:

• “PLR warns us that iSWAP would crash into resources like carriers, other plates”,
• “PLR’s simulator keeps complete track of the iSWAP state, including position and orientation ..”
• “… enabling auto-computation of fastest movement patterns”
• “… enabling automated ‘avoidance zones’ (e.g. to avoid moving over specific plates without having to declare specific paths)”

Call to action

Please review the new iSWAP code, if you have improvement ideas please let us know or submit a PR.

Even though this explainer is already quite long, a lot of detail has been omitted for clarity.

Please ask questions. The more people understand this upcoming PLR feature the faster we can build it and build it even better.

Happy automation

[CamilloMoschner]

teaser of what is coming…

[ben]

@CamilloMoschner

Maybe we are wrong / don’t know the full picture, maybe they are aware and don’t consider it obstructive to most users’ needs

Part of the iSWAP peculiarity is how alignment is handled:

Auto-Adjustment of iSWAP — PyLabRobot 0.2.1 documentation

adjust_iswap.mcr is the Hamilton macro (run via STARService.exe) that uses the iSWAP for conductive probing to map the deck, determine location for the calibration block, set 2 SCARA rotational offsets, and write XYZ cartesian offsets for every unique pose straight into the STAR memory.

I suspect it’s hard to turn those pose-based offsets into continuously variable ones along the full rotational path (you would need to know which of the 2 arms is out of spec), so the firmware just adds the stored offset when it hits each discrete pose (what STARBackend.request_iswap_position() / C0QG returns).

Those pose-based offsets matter if you want the iSWAP to share the same coordinate system as the pipettes for accurate pickup/dropoff at internal deck positions. They are irrelevant for external hotel locations. A fresh iSWAP still needs 1-2 mm offset per pose to drop a plate successfully. Once programmed into the STAR memory they are essentially hidden from the user. Every arm eventually gets crashed (unless PyLabRobot saves us), which adds a few more mm per pose. Once it’s badly bent, adjust_iswap.mcr can’t set offsets anymore.

We should steal an idea from the PreciseFlex400: teach positions empirically by the user. Interactive teaching via firmware (or by hijacking the existing conductive probing pins on every iSWAP) would work. A proper inverse kinematics model makes it easy: arm gets close, you fine-tune the last 10 mm. Combine that with PyLabRobot scripts for quickly teaching external hotels/nonstandard poses and it becomes as straightforward and slightly more time-consuming to teach than a PF400.

[CamilloMoschner]

@ben

I suspect it’s hard to turn those pose-based offsets into continuously variable ones along the full rotational path

My plan was to implement exactly this using interpolation between the calibrated positions

i.e. my calibrated RotationDrive data:

{'home': 13000,
 'left': -29007,
 'front': 156,
 'right': 29068,
 'parking': 29500,
 'extra_1': 29068,
 'extra_2': 29068,
 'extra_3': 29068,
 'extra_4': 29068}

now we can use distance between “left: and “front” / 90 degrees to compute the actual angle forward and backward for a specific pose.

@ben

Once programmed into the STAR memory they are essentially hidden from the user.

…

@ben

hijacking the existing conductive probing pins on every iSWAP

Interesting, doesn’t the iSWAP need that fancy extension to perform probing?

Or can it probe even without it?

[ben]

@CamilloMoschner

Interesting, doesn’t the iSWAP need that fancy extension to perform probing?

Or can it probe even without it?

The calibration kit costs $2.5k, which is very reasonable. If necessary we can use the same pins the service tool uses to engineer something more functional for taming the iSWAP with conductive probing. It is a very simple circuit.