Next developer meeting: march 27 9am PT

Developer Meetings
1

please send me your email if you want to attend

2 Likes
2

bump

1 Like
3

tentative agenda:

  • Discussion of current development efforts. What are people working on / what do people want to work on and what do they need?
  • “State of the union” on capability architecture, arm modeling, liquid handling modeling + implementation work done so far.
  • Future dev plans for PLR / discussion of PLR “wants”
1 Like
4

Any chance you can post here a AI summary (I could ask claude but I want it to be from your perspective and approved) that you find adequate of each of those threads? Some of those threads are super long and intermesh with each other.

Also I know @CamilloMoschner has made some awesome graphics for each option, but I’m a bit lost. Any way you can in these AI summaries link the most up to date graphic for each of these options. Just so we have a nice reference material that everyone’s on the same page with?

Let me know if this is a bad idea and distills too much info for how in depth this discussion will be.

edit: maybe end of day since theres still ongoing discussion

1 Like
5

planning to give the “state of the union” on the threads during the call, will write the notes here after the call.

my thought is that people will want some time to consider/tinker before responding, rather than responding immediately during the call, so just use the call to bring everyone up to date and maybe get some initial feedback. I aim for the next developer meeting (April) to discuss the final implementations after people have commented more / hopefully tested stuff

I will be presenting the v1b1 branch. this is the document I use for Claude to refactor code, it is an “ai / rick” collaboration but still WIP. like the designs there aren’t final. it’s just so easy to do the real thing with claude that the actual branch is my sketch board

1 Like
6

“stop yapping show me the code” GitHub - PyLabRobot/pylabrobot at v1b1 · GitHub

This “meeting” was a lot of lecture/info dumping, to bring people up to speed with discussion and development of v1. Next meeting, I hope people will have gotten the chance to play around and look at the discussion of v1b1. I will be asking people to prepare points to discuss / present next time. I will be testing the v1b1 branch at Retro and brining production protocols over it. I do not recommend using it for production quite yet, but I do recommend people mess around with it so we can discuss finalizing it next week.

Below is state of the union on discussions / meeting notes / explanation of that branch. Please comment in the linked threads. I am closing this thread.

Capability architecture

code first, standardization second

The current “frontends” and “backends” have served us for many years. It has allowed tens of machines and tens of contributors. However there are problems with having LiquidHandler as the interface to all liquid handling machines and Incubator to all incubators, etc.:

  1. LiquidHandler right now is also responsible for resource movement. Problems: a) we are adding standalone arms, which also need the logic of movement, b) some liquid handlers do not have arms so it doesn’t make sense for arm to be on LH → Arm needs to come out of LiquidHandler
  2. Incubator being the resource / entrypoint does not work because incubators look very different. Think SCILA (incubator) vs Cytomat (racks).
  3. Some machines have more than one arm or more than one liquid handler etc.

These problems have become clear as we integrated many machines. It’s no longer scalable or as-easy-as-possible to add new machines (there are 7 different types of machines that do temperature control and only a few share code). Now we can take what we learned and do a next iteration. I think this is such a big change that it warrants calling it “v1”. We have done a ton over experimentation of the last years (and delivered useful code, which is required for people to become interested) - call that the v0 phase. Good enough to scale to 60 contributors, but we are running into problems with it. Now we take that and have the v1 phase. It solves a lot of (hopefully all) problems we currently have, and hopefully will take us from 60 to 600 contributors, at which point we will probably have new bottlenecks and do a v2.

See more background and original proposal in the first post of this thread:

So we want to take Arm out of LiquidHandler. These concepts exists on the same level, Arm, LiquidHandler, etc. We call these Capability. Capabilities are very similar to v0 front ends and backends. In fact, capability follows the same frontend/backend architecture with frontends providing shared logic and nice user APIs, and backends being subclasses of a CapabilityBackend(ABC) to specify the contract of atomic commands.

We don’t want users to instantiate each capability for each machine, which necessitates a higher level object to “orchestrate” these capabilities. We will call this Device. It is the entry point a user instantiates when starting a PLR script:

from pylabrobot.hamilton.liquid_handlers.star.star import STAR
from pylabrobot.resources.hamilton import STARDeck
deck = STARDeck()
star = STAR(deck=deck, chatterbox=True)

A Hamilton STAR has several capabilities, which are actually loaded at runtime! based on machine configuration.

class STAR(Device):
  def __init__(self, deck: HamiltonDeck, chatterbox: bool = False):
    driver = STARChatterboxDriver() if chatterbox else STARDriver()
    super().__init__(driver=driver)
    self._driver: STARDriver = driver
    self.deck = deck
    self.pip: PIP  # set in setup()
    self.head96: Optional[Head96Capability] = None  # set in setup() if installed
    self.iswap: Optional[OrientableArm] = None  # set in setup() if installed

Each capability has a backend. When a Device might have multiple capability backends, they need to share a machine connection, send_command etc. This creates a new concept called the Driver, which provides the shared connection as well as backend methods that do not cleanly fit into a capability backend. I imagine we will put methods like request_iswap_version in the iswap capability backend, even if the backend ABC does not require such a method. General methods like request_machine_configuration will live on Driver.

In diagram:

image
image2484Ă—538 48.1 KB

For complex machines, splitting into capabilities and backends is a natural way of splitting the 10k line god classes and god files.

Since we no longer have LiquidHandler and co as the entry points, we will have them live in manufacturer specific packages:

from pylabrobot.hamilton import X

Re labware, either we will:

  • call this from pylabrobot.devices.hamilton import X
  • move labware definitions out of pylabrobot.resources into top level packages

(Organizing vendor packages in v1)

Tool switching robots / transient capabilities: will only exist sometimes, through context managers.

async with star.core_grippers(front_channel=7) as arm:
  ...

(currently you can call move on star backend even when paddles are not mounted, leading to a runtime error - that’s not expressible in v1)

(Updating PLR API for machine interfaces discussion - #3 by rickwierenga)

Backward compatibility: we will provide a pylabrobot.legacy module which has the v0 API delegating to the v1 implementation.

from pylabrobot.legacy.liquid_handling import LiquidHandler

Makes it super easy to use v1 APIs for some of your code, while still keeping other parts in v0 apis.

(Updating PLR API for machine interfaces discussion)

Capability specific discussions

Now that we are making capabilities, while they are inspired by v0 front ends, they are not identical. So we actually have the chance to make big changes we have came to realize we need. This means we get to rethink how some things are done, which has sparked several discussions:

  • Modeling plate reading capabilities
    • decided: we will split PlateReader into AbsorbanceReader, Luminescence, Fluorescence
    • decided: we need separate methods for kinematics
    • still deciding: how streaming will work
    • still deciding: return format for plate readers
    • figuring out: how we can make plate reader backends work on a coordinate level rather than plate? would like to, if reasonable
  • Modeling arm capabilities
    • ~decided: we need at least three arm classes (gripper necessitates resource_width, orientable adds direction = rotation.z, articulated changes direction into Rotation). each of these parameters is required for that “level of arm” yet has no meaning for a “different level”. Like direction has no meaning on core grippers.
    • decided: we will resolve resource → center-center-center coordinates (for gripper) on the front end, pass those to backend
    • still deciding: addressable space for an arm (deck for iswap and core, undecided for pf/kx)
    • still deciding: how to make teaching in cartesian/resource model as easy as it is in joint space
    • decided: joint space/ joints are modeled through HasJoints, a mixin for backend classes (joint space is something that will never be universalized)
    • still deciding: how can we prevent explosion with CanGrip: each arm type I mentioned can grip or be a point-pickup (like suction). grippers add a required resource_width which does not make sense for non-grippers.
      • will probably need capability backends for each, but on the front end it can be abstracted through the resource model (ie pick up resource X from direction D, capability front end decides whether to compute resource width or not)
    • decided: rails and arms are in one space (Modeling rails in arms)
  • Modeling liquid handlers
    • decided: we will split “independent channels”, head8, head96, head384
    • undecided: what do we call “independent channels”. current frontrunner is PIP.
    • decided: universalize minimum_traverse_height_start, minimum_traverse_height_during, minimum_traverse_height_end (Start/end traversal height parameter names)
    • decided: we will have separate capabilities for warshers and bulk dispensers
    • decided: single channels will be “independent channels” with n=1

BackendParams

We are deleting backend kwargs in favor of BackendParams to

  • make it easy to see which parameters are backend which are frontend
  • make it easy to split parameters in composite methods like transfer and stamp and move_plate (pickup_params, drop_params)
await star.head96.aspirate(
  plate,
  volume=50,
  backend_params=STARHead96Backend.Aspirate96Params(use_lld=True)
)

Conclusion

Please play around with the v1b1 branch!

As soon as it’s ready for more serious testing, hopefully next week, I’ll merge it into main and continue development there (Starting v1 work on plr:main). 0.2 and 0.2.1 are existing versions you can continue to use if you don’t want to touch it, for example in production protocols. We might do a 0.2.2 if we find critical things.

We will discuss these topics more next developer meeting, which is going to be on April 24 - 9am PT.

2 Likes