V1b1 branch merged

General
1

Last weekend I merged the v1 branch. It’s still a WIP towards the actual version 1 beta 1 release, we will continue developing on the main branch until it’s ready for release. The old modules are still available in a new pylabrobot.legacy folder which makes it easy to mix the old and new APIs in your code (just insert .legacy to keep the old API).

per Robot agnosticism and the last dev meeting, the drivers are standalone python classes that have a driver API specific to a given machine. TLDR is the driver layer is probably the wrong layer for making agnostic APIs, as it is adding more complexity than it reduces. We still use PLR-standard units, the resource model, and in general make the APIs as similar as possible without strictly enforcing it.

also, I pushed what was on main last week to a new version 0.2.2 (pypi)

You can see the docs for the current main branch here: User guide — PyLabRobot dev documentation. It’s a WIP but might have some useful information.

I already ported:

  • Agilent BioTek Cytation 1 pylabrobot.agilent.biotek.cytation.cytation1.Cytation1
  • Agilent BioTek Cytation 5 pylabrobot.agilent.biotek.cytation.cytation5.Cytation5
  • Agilent BioTek Synergy H1 pylabrobot.agilent.biotek.synergy.synergy_h1.SynergyH1
  • Agilent BioTek EL406 pylabrobot.agilent.biotek.el406.EL406
  • Agilent VSpin pylabrobot.agilent.vspin.vspin.VSpin
  • Agilent Access2 loader pylabrobot.agilent.vspin.access2.Access2
  • Agrowtek pump array pylabrobot.agrowpumps.agrow_pump_array.AgrowPumpArray
  • Azenta A4S sealer pylabrobot.azenta.a4s.A4S
  • Azenta XPeel pylabrobot.azenta.xpeel.XPeel
  • Brooks PreciseFlex (PF400/PF3400) pylabrobot.brooks.precise_flex.PreciseFlex
  • Byonoy Absorbance 96 pylabrobot.byonoy.absorbance_96.ByonoyAbsorbance96
  • Byonoy Luminescence 96 pylabrobot.byonoy.luminescence_96.ByonoyLuminescence96
  • Cole-Parmer Masterflex pylabrobot.cole_parmer.masterflex.Masterflex
  • Inheco ThermoShake pylabrobot.inheco.thermoshake.InhecoThermoShake
  • Inheco CPAC pylabrobot.inheco.cpac.InhecoCPAC
  • Inheco temperature controller pylabrobot.inheco.temperature_controller.InhecoTemperatureController
  • Inheco TEC control box pylabrobot.inheco.control_box.InhecoTECControlBox
  • Inheco SCILA pylabrobot.inheco.scila.SCILA
  • Keyence barcode scanner pylabrobot.keyence.barcode_scanner.KeyenceBarcodeScanner
  • Mettler Toledo WXS205SDU pylabrobot.mettler_toledo.MettlerToledoWXS205SDU
  • Molecular Devices ImageXpress Pico pylabrobot.molecular_devices.imageXpress.pico.Pico
  • Molecular Devices SpectraMax M5 pylabrobot.molecular_devices.spectramax.spectramax_m5.SpectraMaxM5
  • Molecular Devices SpectraMax 384 Plus pylabrobot.molecular_devices.spectramax.spectramax_384_plus.SpectraMax384Plus
  • QInstruments BioShake pylabrobot.qinstruments.bioshake.BioShake

and of course the new devices:

  • Azenta FluidX IntelliXcap 96 pylabrobot.azenta.fluidx.intellixcap96.FluidXIntelliXcap96
  • Big Bear orbital shaker pylabrobot.big_bear.orbital_shaker.BigBearOrbitalShaker
  • Curiox HT2000 pylabrobot.curiox.ht2000.CurioxHT2000
  • HighRes LidValet pylabrobot.high_res.lid_valet.HighResLidValet
  • KBioscience KUBE pylabrobot.kbioscience.kube.KBioscienceKUBE
  • KBiosystems Ultraseal ePRO pylabrobot.kbiosystems.ultraseal_epro.KBiosystemsUltrasealEPRO
  • KBiosystems Ultraseal XT Pro pylabrobot.kbiosystems.ultraseal_xt_pro.KBiosystemsUltrasealXTPro
  • KBiosystems Ultraseal PRO pylabrobot.kbiosystems.ultraseal_pro.KBiosystemsUltrasealPRO
  • Pioreactor pylabrobot.pioreactor.bioreactors.Pioreactor
  • Sartorius Entris II pylabrobot.sartorius.entris.SartoriusEntris2
  • Thermo Scientific ALPS 300 pylabrobot.thermo_fisher.alps.alps300.ThermoScientificALPS300
  • Thermo Scientific ALPS 3000 pylabrobot.thermo_fisher.alps.alps3000.ThermoScientificALPS3000
  • Thermo Scientific ALPS 5000 pylabrobot.thermo_fisher.alps.alps5000.ThermoScientificALPS5000
  • UFactory xArm 6 pylabrobot.ufactory.xarm6.XArm6

These are just in legacy rn and we are working on porting them over:

  • Hamilton STAR / STARlet (@CamilloMoschner will be responsible for this)
  • Hamilton Vantage
  • Hamilton Nimbus
  • Hamilton Prep (TCP/HARP etc.)
  • Tecan Freedom EVO
  • Opentrons OT-2
  • BMG Labtech CLARIOstar (@CamilloMoschner will be responsible for this)
  • Tecan Infinite 200 Pro
  • Tecan Spark 20M
  • Molecular Devices SpectraMax Gemini EM
  • Cytomat
  • Liconic
  • Inheco Incubator Shaker
  • Opentrons Thermocycler
  • Inheco ODTC
  • Thermo Fisher ATC
  • Thermo Fisher ProFlex
  • Opentrons Temperature Module
  • HighRes MicroSpin
  • Hamilton HEPA fan
  • Hamilton Heater Shaker
  • Hamilton Tilt Module
8 Likes
2

May I know why we are removing the frontend and shared classes in v1? I’m unable to find the reasoning behind this.

I find the v0 architecture very helpful because of its abstraction and shared frontend:

Protocol
↓
Shared frontend
LiquidHandler / PlateReader / TemperatureController
↓
Standard backend contract
↓
STARBackend / OT2Backend / InfiniteBackend
↓
Hardware

That said, v0 does become difficult when a single device exposes multiple capabilities. Because of that, I also agree with the earlier v1b1 direction:

Device
├── LiquidHandler capability frontend
├── Arm capability frontend
├── TemperatureController capability frontend
└── Shaker capability frontend
↓
Capability backends
↓
Driver

This seems like a good direction to me because there is still a shared frontend and capability grouping, while allowing one physical device to expose multiple capabilities. I also understand that migrating from v0 to this model may take some time and effort for existing users and developers.

However, from what I understand, the merged v1 now removes shared frontend classes such as LiquidHandler entirely, and instead relies on loosely consistent API naming at the device or sub-device level:

Protocol
↓
Concrete device or physical subdevice
↓
pylabrobot.io transport
↓
Hardware

I can see how this helps with flexibility and makes it easier to support an increasing variety of devices in PLR. However, I’m concerned that this may come at the cost of direct interoperability between devices with the same capability, similar to what was discussed in the machine-agnostic discussion.

For example, the current v1 design seems convenient for automation within a particular lab where the set of robots is fixed and known. But it seems harder to build higher-level abstractions on top of PLR, or to move a protocol between labs that use different devices with equivalent capabilities.

May I know what the intended final direction is here?

1 Like
3

yes definitely. Essentially what I wrote in this post: Robot agnosticism

TLDR: driver layer is not the right layer for robot agnosticism

That is indeed a cost to this new abstraction. As with every design decision it’s a trade off between costs. I laid out the pros and cons as I see them in the linked thread, and it heavily weighs towards having unique “front ends” per device.

Probably: unique top level interfaces for every device, following PLR conventions (like naming, units, resource model, life cycle etc.), with devices sharing as much code as possible behind the scenes. So if multiple devices do the exact same operation, we should share code between them. For example multiple liquid handling robots having return tips should use a shared helper, arms (and now operators) moving resources should use a shared helper, etc. In the future, I would love to see an abstraction layer above PLR that is working at the level of scientific intent, where it actually becomes possible to write robot agnostic protocols.

2 Likes
4

this makes sense. those working at a higher level will just need a “good” adapter for PLR, which can be defined for their own use cases.

i think this is nice, but PLR needs to make sure that the merged device exposes functions that are atomic for protocol use, for example aspirate/pickup/read_absorbance. just like you said, the “unique top level interfaces for every device”.

when i was writing previous device drivers into PLR, the v0 frontend made it easier to follow the standard. i just don’t want lots of private helpers and low-level functions for the firmware instead of the usually used top level functions. otherwise, it would be hard for us devs to figure it out, as well as for LLMs.

1 Like
5

that’s a good point, what specific pointers were helpful?

I imagine in the future/present we can point LLMs at particular devices and say “this is a comparable device, see how it works”

Also the abstraction levels are always a puzzle for each device, that’s actually a big motivation behind this refactor, in the sense of many devices having much lower level functions that we would like to implement. Frontends were constrained by the common denominator mostly. Writing nice backends (whether with v0 or capabilities) was already a design thing, and in this case it’s just more explicit is how I see it. But point taken on front ends providing good guidelines.

1 Like
6

I think in an ideal world, we’d have both interfaces to the lowest-level abstractions available on any specific hardware, as-well as general higher-level abstractions built in terms of the lower-level ones. However, getting each of those abstractions well-specified is a challenge.

I agree with the direction here that, in the context of hardware drivers and in the time of coding agents, the lowest-level device-specific abstractions are the most valuable ones: Such hardware always remains a black box to some extent. If we try to squeeze a piece of hardware into a shared high-level abstraction, we may forget to specify some subtle details, and end up with leaky abstractions. If we instead forego trying to declare a general abstraction and instead focus on describing exactly what the specific hardware does, we capture the most important information: Framing the semantics of that black-box hardware behaviour. Any entity capable of logical reasoning can then relatively easily combine these low-level primitives into higher-level ones, as long as they can rely on the semantics of the low-level primitives. The converse is not true: If we have leaky high-level abstractions for black-boxes, only entities with access to all the abstracted hardware and suitable test fixtures can change anything below that abstraction, because otherwise there is a high risk of unintended breakage.

2 Likes
7

you are definitely right here. the frontend helps with establishing common ones a device should have when it is similar to an already integrated device. that was how i did it for the tecan infinite. we don’t even need a shared frontend for this if there are already backends for similar devices in PLR. but for a new one like an electroporator, the right way might be exposing the low-level functions as well as good top-level functions.

i’ve also been adding more functions for the tecan infinite now that it is no longer constrained by the “common denominator”. sometimes, the denominator is common but has not yet been added for previous devices (such as state tracking for the tray or sensing plate presence), and this kind of propagates to new devices when we strictly define the frontend (or just don’t have time to look through every device type).

8

thanks for the work and that’s a good example! in the past it was already possible to add things to backends, but it was a little odd to use it

I would say from that perspective little has changed then