Helper classes: lifecycle hooks and access pattern

[vcjdeboer]

@rickwierenga

See this file pylabrobot/pylabrobot/hamilton/liquid_handlers/star/misc/architecture.md at v1b1 · PyLabRobot/pylabrobot · GitHub for an overview of the current new architecture. Code is on that branch next to that file.

Looking at the STAR architecture doc, the split between capabilities and helper classes is really clean. Nice pattern. Own files, own tests, clean classes. And they’re naturally ready to be promoted to full capabilities when other devices turn out to have similar subsystems (e.g., another device gets an autoload → extract an AutoloadBackend ABC).

Two questions:

1. Lifecycle. Capabilities participate in teardown via _on_stop(), the shaker stops, the heater deactivates, all before the connection closes. The helpers don’t. If stop() is called while the wash station is filling or the cover is locked, they don’t clean up. Should helpers have _on_stop() hooks even without being full capabilities?

2. Access. From the architecture doc:

# Subsystems — on the driver
 await star._driver.autoload.load_carrier(carrier_end_rail=10)
 await star._driver.cover.lock()
 await star._driver.wash_station.drain(station=1)

Loading carriers and locking covers are user operations, but they’re behind _driver (private attribute). If they lived on the Device, star.cover.lock() instead of star._driver.cover.lock(), that would also solve the lifecycle question, since the Device already manages _on_setup() / _on_stop() for everything it owns. But maybe
I’m missing why they need to live on a separate _driver attribute?

[CamilloMoschner]

Very good points @vcjdeboer !

I don’t know where to ask this question because there are so many threads, but seeing this here now:

Why would _driver be marked as private?

This indicates that it should not be used by PLR programmers but every aP I use requires access to driver-specific features:
No Capability abstraction can cover every unique feature of a device. But we in the lab purchase device A over device B because of device A’s special features.

As a result, I cannot imagine a production aP that would not require the PLR programmer to access the currently marked “private” driver.

---

[rickwierenga]

@vcjdeboer

The helpers don’t.

good point, I missed that.

Should helpers have _on_stop() hooks even without being full capabilities?

definitely, 100%. Simple oversight

@vcjdeboer

_driver (private attribute).

yes driver should be a public attribute for sure

@vcjdeboer

that would also solve the lifecycle question, since the Device already manages _on_setup() / _on_stop() for everything it owns. But maybe
I’m missing why they need to live on a separate _driver attribute?

I considered this, but ran into some problems:

• which object should own these subsystems that are not capabilities: Device or Driver?
• I don’t like Device owning universal capabilities on the one hand, and subsystems (necessarily device specific) on the other hand. Driver is naturally specific to a device
• I think the Device functionaries should be accessed through either capabilities or drivers. Device.do_something really mixes the responsibilities. I think Device should only manage its resource model, capabilities and its driver.

[vcjdeboer]

@rickwierenga

I don’t like Device owning universal capabilities on the one hand, and subsystems (necessarily device specific) on the other hand.

I think the word “subsystem” is creating a false category. star.iswap.pickup_plate() and star.absorbance.read() have the same shape, device.thing.action() or like before device.attribute.method(). Both need lifecycle hooks. Both are user-facing. The only difference is whether a shared ABC exists yet. A Cover is just a capability that only one device has today. Doesn’t that make them structurally the same thing?

@rickwierenga

Device.do_something really mixes the responsibilities.

Agreed. But device.cover.do_something() doesn’t mix anything, the Cover is a named, scoped object with its own lifecycle. The Device isn’t doing the thing; the Cover is. Isn’t the scoping through the attribute what keeps responsibilities clean?

@rickwierenga

which object should own these subsystems that are not capabilities: Device or Driver?

I think everything user-facing, device.iswap.pickup_plate(), device.cover.open()is a capability, and the Device owns it. Everything internal, USB connection, command serialization, firmware protocol, internal state; is the driver. What’s the concrete example of something user-facing that should not be a capability?

@rickwierenga

I think Device should only manage its resource model, capabilities and its driver.

I think we agree. iSwap, head96 those are attributes that hold capabilities. So the Device manages resource model, attributes holding capabilities (universal and device-specific, all with lifecycle hooks), and the driver (internal, not user-facing).

Cover is a special helper case, a device-specific capability without an ABC, but I would still make it an attribute that holds a capability with lifecycle hooks.

[rickwierenga]

@vcjdeboer

Doesn’t that make them structurally the same thing?

it makes it the same thing as a CapabilityBackend in my opinion, which are not accessed from Device directly.

Agreed on life cycle hooks, it should mirror CapabilityBackend without actually being one.

@vcjdeboer

the Cover is a named, scoped object with its own lifecycle

what if we actually add a cover capability in the future?

@vcjdeboer

What’s the concrete example of something user-facing that should not be a capability?

capabilities are defined to be universal, and autoload/cover/wash stations right now are not universal. they are specific to a star. The driver is still doing the internal things you mentioned (+ some commands that do not clearly fit into a subsystem or capability backend like request machine configuration)

[vcjdeboer]

@rickwierenga

capabilities are defined to be universal, and autoload/cover/wash stations right now are not universal.

That makes sense. Maybe then it should get its own type class, instead of just a random class on the driver. What if we call them

Localities ?

Local to one device, in contrast with capabilities that are global across devices.

  class Locality:
    """Device-specific functionality with lifecycle hooks."""
    async def _on_setup(self): pass
    async def _on_stop(self): pass

  class STARCoverBackend(Locality):
    def __init__(self, driver):
      self.driver = driver
    async def lock(self): ...
    async def _on_stop(self): await self.unlock()

On the Device, same list, same lifecycle:

  self.pip = PIP(backend=self._driver.pip)          # Capability
  self._capabilities = [self.pip]
  self.cover = STARCoverBackend(self._driver)       # Locality
  self._capabilities.append(self.cover)

User sees star.cover.lock(), same pattern as capabilities.

@rickwierenga

what if we actually add a cover capability in the future?

That would be the promotion path. A Locality matures into a Capability when a second device needs the same functionality. You extract the ABC, the Locality becomes a proper backend, user code doesn’t change.

Alternatively, if a separate Locality base class feels like too much, the Cover could just directly inherit from Capability without an ABC. When a second device needs Cover control, you add the abstract CoverBackend at that point.

[rickwierenga]

@vcjdeboer

Maybe then it should get its own type class, instead of just a random class on the driver. What if we call them

Localities ?

Local to one device, in contrast with capabilities that are global across devices.

Since they are necessarily device specific, otherwise they would be capabilities, I am hesitant to add a PLR class for it. All this class would do is specify _on_setup and _on_stop, which I don’t know if that’s worth it.

@vcjdeboer

A Locality matures into a Capability when a second device needs the same functionality. You extract the ABC, the Locality becomes a proper backend

Conceptually I agree

user code doesn’t change.

It might to the extent that what you call “localities” (maybe having a name is useful ) are more analogous to capability backends rather than capabilities. the capabilities might abstract some of the convenience methods and stuff, which might be implemented on the locality before that, or they might be new convenience methods.

[vcjdeboer]

@rickwierenga

Since they are necessarily device specific, otherwise they would be capabilities, I am hesitant to add a PLR class for it. All this class would do is specify _on_setup and _on_stop, which I don’t know if that’s worth it.

I think it is worth it. The lifecycle hooks are our first contract with the physical world, _on_stop is what unlocks the cover, parks the autoload, drains the wash station. If these don’t close properly, hardware is left in an unsafe state. That alone tells me it needs a proper class, not something ad-hoc on the driver where the hooks can be forgotten.

It also keeps coming up across devices. STAR has cover, autoload, wash station. Cytation has the tray. Every device has these local things that need lifecycle but aren’t universal yet.

But that’s just my take.

Other solutions to make sure they get lifecycle are also possible of course.

[rickwierenga]

@vcjdeboer

If these don’t close properly, hardware is left in an unsafe state.

well yes, but how does a locality class help with that? it is still up to the main driver class to call it.

[vcjdeboer]

@rickwierenga

well yes, but how does a locality class help with that? it is still up to the main driver class to call it.

Not the driver, the Device. The Device already iterates _capabilities and calls _on_stop() on each one in reverse order. That’s how PIP, Head96, and iSwap get their lifecycle today. If cover is in that same list, it gets the same treatment automatically.

But cover can’t be a Capability, capabilities are universal by definition, and Capability.__init__ requires a CapabilityBackend. There is no CoverBackend(CapabilityBackend) ABC for it to wrap. Making backend optional would be needed then.

So cover needs to be in _capabilities for lifecycle, but can’t extend Capability to get there. That’s the gap a Locality class fills, it gives cover the lifecycle hooks and a type that belongs in _capabilities, without changing Capability itself.

[rickwierenga]

@vcjdeboer

If cover is in that same list, it gets the same treatment automatically.

but it’s not since localities are backends/ driver level and capabilities are device level

I don’t think device should be managing localities. (last comment in Helper classes: lifecycle hooks and access pattern - #3 by rickwierenga)

[vcjdeboer]

@rickwierenga

but it’s not since localities are backends/ driver level and capabilities are device level

I don’t think device should be managing localities. (last comment in Helper classes: lifecycle hooks and access pattern - #3 by rickwierenga)

I want to make sure I understand your model correctly. In your architecture:

• Device.stop() iterates _capabilities in reverse order, calls _on_stop() on each
• Cover, autoload, wash station stay on the driver

How does Driver.stop() handle lifecycle for these? Right now it sets them to None. If they get _on_stop() hooks, does the driver iterate them in order too? And how does the ordering work between the two, do all capabilities stop first, then all driver helpers? Or is there a way to interleave them?

[rickwierenga]

@vcjdeboer

How does Driver.stop() handle lifecycle for these?

there are two options:

• device handles lifecycle for all capabilities, driver for all capability backends + localities. capabilities do not handle capability backend life cycle
  • pro: drivers can be used standalone without device
• device handles lifecycle for all capabilities, and capabilities for capability backends. driver only does life cycle
  • pro: works nicely when people are using capabilities outside of devices, like I imagine will be the case for testing