Robot agnosticism

General
1

I am having thoughts about building PLR to have a universal api between machines.

We rarely benefit from this:

  1. machines are often not exactly the same, so the abstraction is necessarily leaky or you hurt power over machines
  2. changing out the physical hardware is already some amount of work
  3. claude can easily rewrite the code to use a new api, this is essentially 0 effort (esp compared to (2))

where it does help:

  1. some machines like heater shakers are like exactly the same
  2. for plate readers, you probably want to have a standard data format. similar for other analytical machines. (but this can be loose rather than rigid)
  3. for plate definitions and the resource model in general, sharing that is actually useful
  4. certain features like volume tracking are best shared?
    • we are starting to have different liquid handling devices, like echo/idot/mantis etc. which are liquid handlers but do not use the liquid handling capabilities. It is already the case that we have multiple dispense functions. I haven’t figured out how to make them nicely share this call, so wondering if it’s the right abstraction
  5. the io layer is pretty nice to share

obviously we do benefit from sharing code between machines.

However, it hurts the user API if we enforce universal apis. For example, it introduces complexities like “backend kwargs” (v0) or “params” (v1).

In the past when writing code was more manual and annoying, it made sense to want to switch out backends. Nowadays this doesn’t really hold up anymore. Claude/etc. is probably our #1 user so we might as well design the API for them

Higher level: is the driver layer (essentially what PLR is) is the right place for abstraction among devices? it might be something for the protocol layer/scientific intent.

2 Likes
2

Currently we implement convenience methods like pierce_foil in the STAR backend because it involves movements that are not possible on the OT-2, and will be vastly different when implemented for Tecan and other liquid handlers with different “backend kwargs” / “params”.

Most lab automation workflows are physically complex and can fail to replicate because of small differences in the choice of backend parameters. Sequences of basic steps can be vastly different given the same scientific intent on physically different instruments.

Scientific intent, like primer aliquoting, magnetic purification, and suspension organoid media exchanges, often involves a sequence of multiple steps. Across liquid handlers, PCR, and qPCR machines, implementation details change vastly depending on the individual volumes and total throughput required by the assay.

However these machines do have fundamental similarities - most PCR workflows can be run on a qPCR machine with the added benefit of quantification in parallel with amplification. Ideally PLR makes it obvious to the user if they can use existing equipment, or if they have to buy something new to solve their problem.

Someone might go as far to buy an automated roll heat sealer when they can totally just use mineral oil instead.

When a user has tested backend parameters and established reliability for their process, we can reduce replication costs by sharing the complete command set and reagent details with other users. We haven’t yet seen PLR users share their process at this level of detail. At my last startup (Retro) sharing such a complete example breaks internal confidentiality rules because it reveals what we are working on at a deep level.

Universality among the most basic steps like aspirate, dispense, move_plate maintains secrecy about the larger process, but fails because this type of universality is not useful to an end-user with just 1 type of machine in their lab.

The best solution is likely an abstraction among devices at the protocol layer, but this will only be effective if we are able to share deep implementation details without being so high-level as to reveal exactly what is going on.

3 Likes
3

I really like this, and honestly I’ve been sitting on a version of it for a while. I built an abstraction layer that connects to PLR legacy or v1b1. What I didn’t fully realize until I read your post @rickwierenga is that the architecture itself is getting less important. As long as the machine is actionable from Python, who cares whether the folders are organized by vendor or whether the instrument class goes through the capability layer or straight to the driver :slight_smile: .

I’m not sure this is exactly what you mean, but the thing I keep coming back to is that the contract between the protocol layer and PLR needs to be fixed. That’s what I built Patch for. I took the idea from MIDI (the way synths and other music devices have talked to each other for decades). The idea: a hashable bytestring that encodes the PLR commands. The protocol layer builds it, and it carries the whole context: labware, reagents, protocol, instrument, PLR version. A “player” translates it back into PLR, and PLR executes. The player is the translation seam, and it doesn’t have to be PLR or Python, any driver type or language can sit behind it, as long as the player stamps the definitions it runs against (labware, resource model, versions). The bytestring is the content-addressed unit; the abstraction lives at the intent, not at the driver method.

I built a first version as a swamp extension

I’ve been building in Swamp the last few weeks and it feels like the right way forward for me: deterministic AI models and workflows. I’ve used it for data analysis pipelines and provenance work.

This is the hard part indeed. Those per-assay details are exactly what usually doesn’t get captured. We might now be able to pull more of what’s in the lab protocol into a deterministic form. And there are opportunities for sharing Patches (within academia, for example) or keeping your own local collection.

1 Like
4

i think this is helpful and kinda wish a community would work on this. it feels like devs have always wanted this layer between a protocol and PLR atomic commands, and several papers have tried to make it, but it never ended up becoming the standard. somehow, though, i think PLR is capable of doing it now since it has gotten good at the lower level.

previously, i was unsure whether this would be of interest to PLR, since the scope before seemed to focus more on the atomic command level. i vibe coded one repo for this for my own use—would this be similar to what you are referring to?

basically, it provides higher-level commands at the step level, which is usually how experimentalists read a lab protocol. i always feel that an LLM has a tendency to miss something crucial when working at the lower level of executing lab steps, so wrapping them into a good higher-level step makes them more reusable and safer.

in my repo, PLR is an adapter, and i organized these as nodes so they can be connected modularly. this was inspired by XOD, the no-code IDE for Arduino. connected nodes can also be assembled into a higher-level node, such as a recipe, i.e. a protocol.

so, for example, one recipe might involve active cooling. as long as the devices in PLR support it, it should work regardless of whether the device is a heater-shaker or a thermocycler. this also makes it more portable for sharing, though, as you said, coding agents can now easily rewrite the code to match the instruments we have in our lab.

(base) […] % tree modulap/nodes
modulap/nodes
├── init.py
├── _composition.py
├── _manual_placement.py
├── _request_hooks.py
├── _specs.py
├── imaging
│ ├── init.py
│ ├── calibrate.py
│ ├── map_pixels.py
│ ├── model.py
│ └── specs.py
├── liquid_handling
│ ├── init.py
│ ├── _support.py
│ ├── aliquot.py
│ ├── channel_policy.py
│ ├── combine.py
│ ├── dilute.py
│ ├── formulate.py
│ ├── jog.py
│ ├── mixture_formulation.py
│ ├── model.py
│ ├── prefill.py
│ ├── request_hooks.py
│ ├── runner.py
│ ├── serial_dilution.py
│ ├── specs.py
│ ├── spotting.py
│ └── transfer.py
├── meta
│ ├── init.py
│ ├── assert_step.py
│ ├── init_liquids.py
│ ├── log.py
│ ├── mark.py
│ ├── notify.py
│ ├── pause.py
│ ├── request.py
│ └── wait.py
├── movement
│ ├── init.py
│ └── labware_manual.py
├── plate_reading
│ ├── init.py
│ ├── _support.py
│ ├── absorbance.py
│ ├── capture.py
│ ├── fluorescence.py
│ ├── luminescence.py
│ ├── model.py
│ ├── read.py
│ ├── runner.py
│ ├── specs.py
│ └── tray.py
├── recipes
│ ├── _golden_gate_support.py
│ ├── _request_orchestration.py
│ ├── _transformation_support.py
│ ├── golden_gate_program.py
│ └── transformation_heat_shock.py
├── runtime
│ ├── init.py
│ ├── _support.py
│ ├── deck.py
│ ├── device.py
│ └── specs.py
├── sealing
│ ├── init.py
│ ├── model.py
│ ├── runner.py
│ ├── seal.py
│ └── specs.py
├── separation
│ └── init.py
└── thermal
├── init.py
├── _support.py
├── _waiters.py
├── cool.py
├── heat_shock.py
├── incubate.py
├── model.py
├── presets.py
├── program.py
├── runner.py
└── thermocycle.py

11 directories, 76 files

1 Like
5

i agree the abstraction should be at this level. i wrote my reply here as well:

additionally, regarding what you referred to about the problem of non-reproducibility caused by different backends across users and machine setups, i do think this is also a universal problem with human hands.

what i mean is that the problem is the same, but with machines, we can now reduce the tacitness because we can log what happens. in the repo i mentioned, where i use nodes, they are hierarchical and can also log hierarchically: recipe node → step nodes → PLR atomic commands.

i think this is also similar to how we already troubleshoot with humans. when a senior checks how a junior did a protocol, it is much clearer to first check the overall recipe, then the individual steps, and only then go down into the lower-level details. it is much harder to understand what went wrong if everything is shown as one flat list of PLR atomic commands.

for example, say there is a recipe for purifying DNA using magnetic beads. someone could share the recipe, and i could use it with my own machines. if something goes wrong, we could more clearly see whether the problem is at the recipe level, the step level, or the backend level.

from there, we could tweak the recipe to make it more general, document which machines are compatible or incompatible, or make different recipes for different sets of machines. as long as they work.

1 Like
6

What about starting at the most common and universal substeps, like one PCR prep, one ELISA, etc. I feel most end users, especially in academia, are not looking to automate everything end-to-end like samples in > data out. Automating the routines can already significantly decrease hands-on time and help reproducibility. No IP risk either.

Then we make the substeps modular and configurable so they can be connected together. Standards must be rigidly enforced somehow tho, even when there may be waste/inefficiencies. There are a million ways to do PCR but they all accomplish basically the same thing. Since PLR is already so universal and easy to develop with, we may be able to just assemble the protocols, and if enough people use them because of PLR’s accessibility, they’ll become the standards.

1 Like
7

at this point i am not sure if it should live in PLR, just that the abstraction should probably live on that level of, regardless of where the code is.

:100:

2 Likes
8

I think the question might be the wrong one.

With the v1 capability architecture we already went device-centric:

byonoy.luminescence.read(mode="custom", integration_time=100)
clariostar.luminescence.read(focal_height=10)
star.pip.transfer(...)

No universal API. The STAR has a wash station; the Byonoy has no focal height. Not leaks in an abstraction. Nothing to leak from.

Your move is: the LLM rewrites the driver for free (point 3), so the driver doesn’t need a universal API. Push the abstraction up, to the protocol / scientific-intent layer.

But up is the same problem. “Read luminescence on the plate” is clean and universal and it doesn’t run. The moment you make it runnable you’re choosing focal_height or mode, ClarioStar or Byonoy device-centric again. Universalize that and it leaks; keep it device-centric and nothing was abstracted, you’ve just moved the same code up a floor. Where the abstraction lives doesn’t change anything. It’s the same at every layer.

So “where should the abstraction live” has no good answer, because it’s the same answer everywhere. That’s the tell it’s the wrong question.

Here’s what actually changed. The old universal API was leaky but deterministic. Whatever writes the device-specific version now (probably the LLM, at whatever layer) is clean but not deterministic. Ask twice, get two protocols. Both plausible, both run.

So what we’re missing isn’t abstraction. It’s determinism.

A protocol on its own is under-determined, “transfer from role 1” means nothing until something says what role 1 holds. Seal it to a resource model and a reagent model and it becomes a fixed, reproducible experiment. Check that sealed thing against the same models. Bindings resolve, volumes stay feasible, and it will run.

Which lands back on your list. Resource model, volume tracking, plate definitions, you have them as shared code. I think they’re the models you seal and check against.

Yes that is why you need the determinism, not a higher-level step.

1 Like
9

The point of capabilities was to make them universal. With this, I am essentially proposing to make everything a locality (the term you suggested). Localities between different machines might/should look similar, but this would no longer be enforced.

I think this is close but not quite accurate. If you are looking for focal_height, as you would on the driver layer, yes that is never gonna be solved. However, on the higher levels you probably dont care about this, you care instead about things like “absorbance of a given well in a given plate”, and then focal height is just an implementation detail (an important one, but really at that point you care about getting absorbance and not about how you got it as long as it’s accurate and precise)

10

Fair point. focal_height was a weak example. Take gain instead, or integration time.

Gain isn’t an implementation detail, because the correct value depends on your samples, not on your device. The gain that works for a dim luminescence assay saturates a bright one. The driver can’t pick it: it doesn’t know your assay. And it can’t be pushed up either, gain on a ClarioStar isn’t gain on a Byonoy, the numbers don’t transfer. Device-specific and scientifically load-bearing. That’s the class that breaks the implementation-detail move.

What guarantees that? If the answer is autogain, then the device picks the gain, per plate, from the signal it sees, which means two runs of the same high-level intent get different gain and produce non-comparable data. Now you have to record what it picked.

So either the scientist sets it, or the instrument sets it, and either way it ends up in the record. You can hide it from the author. You can’t hide it from the record.

Which is the point I’m making: this is the class of parameter that has to be sealed, not abstracted.

11

exactly, which is why it needs to exist on the protocol level.

is a specific gain something you want to set explicitly or is it just something to get what you want (good pictures)? I think with objective size / imaging mode, it’s clearly something to set explicitly. And if a machine does not support that, it’s game over. But for gain/exposure time/focal height etc. I imagine in many cases as long as it’s logged and the pictures look good, it’s actually acceptable. Yes the images will look differently on different machines, but that’s already the case. The way to deal with it is with positive/negative control and measure relative to those - protocol level ideas. But yes if you want perfectly reproducible data without having to do processing, then you’d have to use the same machine (or at least same type) and at that point you dont even care about hardware agnosticism at all.

3 Likes
12

agree. this is also why i think it’s good to move the level up, but still allow some flexibility for users. they are also aware and have control of their own runtime. i dont think the objective is to ensure those higher-level functions to work in the sense of, their results work as it should. but just to make it valid, safe and correct steps (like how an intern should do, instead of keep failing at the most basic level and we need to oversee constantly, ie less trust).

llms or coders can still tweak and iterate themselves to get what they want scientifically. but at least we dont have to install lots of skills or provide lots of tutorial examples on how they should use the lower levels correctly. this way, PLR can move towards applications more and higher.

2 Likes
13

I think moving up is really just reinventing the orchestrator layer. But I’d argue PyLabRobot doesn’t need to go up there. The orchestrator level is crowded: John’s FluentAgent (link), @hazlamshamin’s node/recipe layer, and others. I think this work belongs above PyLabRobot rather than inside it. PyLabRobot’s uniqueness is the driver level and the resource model.

Earlier I had the feeling that moving away from a universal API was a good thing. I still do, and it’s very much how things are already going. The new instrument drivers are written by the LLM anyway, and they conform loosely to the capability structure. Nothing wrong with that.

And we can keep the capability and locality structures side by side, or, taken to the end, move everything to localities and go fully device-centric. Either is fine, because the point of a capabilities architecture isn’t hardware-agnosticism anymore (clear from this thread). It’s structure, a consistent shape for the LLM to write in. Capabilities where devices genuinely line up; localities with typed params as the escape hatch where they don’t. And if everything drifts to a locality, that’s okay. The durable value is the organization, not the universality.

But aiming for abstraction at another level, I don’t think that’s PyLabRobot’s job. Let’s instead make the resource model, volume tracking, and clash prevention better, in a real open-source, community-driven way. That’s the layer that makes all these LLM-written drivers actually safe to run, and unlike the orchestrator layer, it’s uniquely ours to get right.

1 Like
14

i agree that it might not be appropriate to live inside PLR as is now. i also was not arguing that it should live inside PLR. however, i don’t think that it should be as high level at the orchestration level especially at agent level. like you said, it will be very messy; higher is messier. hence i think, just slightly wrapping PLR atomic commands (despite “slightly” here actually will involve deep decisions since we are now on unifying and agnoticism).

we are unsure yet where it should be built. but, since it likely will use PLR as an adapter and wrap it, plus many PLR devs and users also likely is going to use and need it, we might want to decide whether this is worth building together? i agree that the lower level stuffs in PLR should stay and keep improving over time too.

15

So the most difficult part of building higher abstractions seems to be ensuring that catastrophic failures are unlikely in the physical world, so that the iteration loops/autoresearch of coding agents can be transplanted to the development/optimization of assays from broad instructions. Coding agents are successful because they allow iterations to fix small errors that are inherent to these models. As long as LLMs are good at getting things close enough, deterministic tests and performance targets can make the final solution approach perfect.

This sounds like a job for digital twins based on PLR’s labware/deck/instrument geometry definitions. Certainly interested in exploring this direction.

1 Like
16

Yes exactly, love that

17

Yes, and I worked it out a bit further, because I think the evidence backs it all the way and the pieces land cleanly. Full writeup with working code for four devices (three readers + a Hamilton STAR):

The short of it:

The abstraction is already being worked around, not grown.

The capability layer is a one-author, one-push artifact, 19 capabilities + 8 localities, built for the devices being migrated. In the three-plus months since, not one new device has extended it. New devices are pure consumers: they reuse a capability and bolt on a BackendParams when it doesn’t fit, or when they genuinely need a new abstraction, stall in review or land with the exact anti-patterns it was meant to prevent (workflow-on-frontend, driver-mirrors-capability, fused backends, **kwargs, duck-typing). It’s what LLM-authored device code looks like: the model writes a locality trivially and struggles to abstract a capability, often because the abstraction isn’t there. A CLARIOstar absorbance read carries device-specific knobs (an OD-vs-transmittance report, focus height) that a Byonoy Absorbance 96 simply doesn’t have. Same intent, no shared parameter set, so a “universal” read either drops what one needs or smuggles it back through backend_params.

“Similar but not enforced” is exactly a tag.

Make everything a locality and organize with a tag, a Value Object in DDD Evans’ sense, a str you search, not a type you dispatch on. It carries the cross-device intent (findable, every absorbance read, every transfer) without enforcing a mechanism that leaks. In DDD terms: a capability is a Port, a locality is a Bounded-Context term, a tag is a Value Object.

This falls right back to this:

Skip the capability and those stop being an abstraction problem and just fall into place: a sub-device is simply a resource. pip, head96, iswap are physical things you can point at, so they live in the resource tree, not in a capability. Each carries its own methods as localities star.pip.aspirate(...), star.head96.stamp(...) the exact call sites we use today, now hanging off a resource instead of a capability frontend. Measurement domains become quantity: facets, coordinators become orchestration, and nothing gets renamed: star.pip stays star.pip; we just stop treating it as a behaviour contract and let it be the sub-device it already is. The gist works this all the way through, up to the STAR.

One more benefit: tip and volume tracking are facts about physical space, not behaviours. Today they’re stuck inside the capability (the PIP capability literally owns the tip trackers). Drop it and they move onto the resource: one deterministic source of truth for where every tip and volume is, instead of each capability tracking its own way.

18

I agree, trying to make it universal with such different hardware is either going to make passing parameters hard or just creating a million capabilities which is no different than just creating drivers for each. This honestly might be headed to where failings of SILA have occurred and why when you bring up a universal architecture for lab auto most people in the industry roll their eyes. We’re not CNC machines we didn’t standardize originally and do far to many things instead of just “removing material”. Three years ago I would have argued we need the universal but now claude can just figure it out. Makes me a bit sad, it would have been cool to solve but probably not necessary.

I don’t think we’ll ever get to the level of abstraction of “PCR reaction” as its just too different for every lab, process, sample, etc… We run the risk that Hamilton and other companies have already fallen for, making “smart steps” with a million parameters that eventually never work for a process and are abandoned in favor of fully controllable steps. Sharing these protocols would solve this and it should not live in the PLR level, you’ll just end up with “smart steps 2.0”. If these protocols were shared you just grab the code, ask claude where it differs from your current SOP, and modify it. No need for this to live in PLR. Which makes me think, is this where capabilities is headed?

As for how to go forward or structure this change, I am unsure.

3 Likes
19

If anything doesn’t this prove the opposite? That the capabilities right now extend to most devices in library. Also 3 months is not that long for an entire switch of architecture for labs already running V1, we got samples to run. Then again I haven’t paid attention to how new devices have been integrated since we’ve still got a lot of V1 going…. Which devices that are new are bolting on BackendParams?

2 Likes
20

I wasn’t able to attend the entire developer meeting but the gist I got was that most found v1b1 a bit confusing?

I had thought driving into work today, what if instead of making all liquid handlers agnostic you only make ones with the same pipetting technology agnostic. As in all air displacement instruments share commands, all fixed tip instruments, etc… I say this as it would be real nice to instantly transfer a Hamilton method to a Tecan.

3 Likes