Hey everyone! I guess this is the place for PLR development now.
I would like to propose “flattening” the resource hierarchy for “common” resources in PLR. These common resources are tip racks, tube racks, well-plates, tubes, tips, and maybe holders.
This is what we have now:
Well Plate → Well
Tube Rack → Tube
Tip Rack → Tip Spot → Tip (not a resource)
Some of these may be in a “holder” or not.
From my point of view, this pattern is inconsistent, and as such it makes code harder to understand and develop for and around. For example, it is very hard to write a parser for PLR’s serialized JSON data; the only easy solution being to use PLR itself, which unfortunately is only available in python.
I also have concerns around the current implementations of decks and backends that I would like to get into, which further complicate the task, but lets keep it simple for now.
What I would like to propose is enforcing a simple, flat pattern:
The “flexibility” of assigning anything as a child of anything is not evil, but it does not make sense in 99% of imaginable cases. You can place a well plate in a tube spot, for example.
My proposal only requires:
Making Tips a Container resource.
Adding TubeSpots to TubeRacks, to hold Tubes.
Adding WellSpots* to Plates, to hold Wells.
Adding ColonySpots to petri dishes, to hold colonies.
…
Adding checks to enforce relationships between parents and children of these types, ensuring consistency.
Refactoring everything that might need refactoring to accommodate this.
WellSpots don’t exist physically, but I don’t think that it should matter. The pattern here is that “spots” are virtual places where stuff can be, in that case a Well, even if it is welded to the plate.
What’s important for this case is the consistency in the usage and extension of PLR.
I have already “forked” PLR to accomplish this with TubeRacks, and make it compatible with my setup. It has been exhausting, unfortunately, and I don’t have the energy to code PRs for everything that needs to happen for a merge.
So lets start here. I would certainly appreciate it.
I honestly don’t see how adding additional constraints makes things simpler instead of more complex. We would have to add checks for types everywhere, which is an artificial constraint and also requires a lot of extra code.
I would argue that the flexibility adds something, even if it is the 1%, but probably more than that. We regularly use custom setups
Could you please elaborate?
I don’t understand what you mean. This looks exactly like what these resources are in physical reality which means that anyone developing with these resources just has to look into the lab to understand what they mean.
What do you mean by "holder"?
I can see how a TubeRack can be seen as a “holder” but don’t understand what else you might mean.
Do you mean you want to force a 5-level resource hierarchy on every resource?
How does this work for carriers, mfx_modules, extended mfx_accessories, plate_adapters, specialised equipment (e.g. a vacuum-extraction system with its various nested holders and collection_containers)…?
All of this is currently possible to represent and easily work with in PLR, precisely because PLR mirrors physical reality.
I disagree. If a resource does not physically exist it is incredibly difficult for developers to figure out what it does, and I cannot see how a resources capabilities can actually be used if its reality is not represented in its code?
This also creates an array of further problems from what I understand that you mean so far:
e.g. how do you make it compatible with movements, how do you visualise it in the deck tree?
I agree that consistency is important and we should try to be as consistent as possible. But if the desire for computational consistency comes at the cost of neglecting physical reality, then it will simply constrain reality.
Ultimately though, I am interested in knowing more about precise instances that you found the current resource hierarchy and its adaptability not working for you?
…and in this regard, what are these "99% of imaginable cases in which the current system doesn’t make sense?
Once we know that we can fix it
I gather that perhaps I’m a bit shortsighted because I don’t understand your contexts and perspectives, or I’m confused about what PLR really aims to be, but I will try to answer the questions I understand.
Rules make things simpler, more predictable, and more reliable. Its the same reason why there are typing hints everywhere, and probably why there is strictness checking for the liquid handler.
What do you mean? I see it as a matter of intercepting assign_child_resource in the most common classes, to check the types of the incoming resource.
Well plates should only hold wells, right?
Remember when I asked about what the Z-coordinate of the TipSpots meant? This kind of question pops-up everywhere, and to write for PLR I would like unambiguous, well documented, fixed answers.
The serialized data is irregular, everything seems valid, and no schemas are defined for it, making it hard to know what the keys and values mean, at each step of the way.
This is the main reason why writing a backend and deck for PLR was hard for me.
What I meant by [n-RackSpots] was “any number of intermediate holders here”, from 0 to N.
I chose the word Holder, but it could have been Carrier, Slot, or Anchor: “a thing that holds a certain other thing in place”.
It also can mirror anything that nested lists of relative coordinates can.
Adding some constraints is reasonable: it will enforce physical relationships between things in software, make it easier to extend, at zero cost for flexibility.
I’ll stand by this point unless I find out that adding TipSpots directly to a deck is common practise in some scenario.
I don’t understand why you wrote this part. I’m not neglecting reality.
In the case of my proposal, I’d like to see the following pattern enforced for all common classes, such that the “useful” object is accessed consistently in PLR and in the serialized JSON.
# Pseudocode
rack_of_some_type = deck.get_resource("any type of rack") # traverses through all holders
container = rack_of_some_type.spot_of_some_type.get_container()
Instead I have three patterns, one for each of the common types, and maybe others I don’t know about yet.
# Pseudocode
tip_rack = deck.<any level of nesting here>
tip_spot = tip_rack.tip_spot # two levels, the third is not a resource
tube_rack = deck.<any level of nesting here>
tube = tube_rack.tube_spot.tube # three levels
well_plate = deck.<any level of nesting here>
well = well_plate.well # two levels, there is no third
I’m just trying to make things simpler… the more PLR code I read, the more special cases I find undermining its agnosticity. But that’s a topic for another post.
I appreciate this feedback. I really aim to make things better.
I will say I fundamentally believe standardization follows implementation. Specifically because of discussions like these. We have an implementation, people use it, it’s not perfect, we update it based on new information, and ultimately we converge. I am sorry we can’t provide a finalized description right now.
I would argue that there is more complexity.
Taking “plates should only take wells” as an example. Yes, they should. This is implemented through the generic type ItemizedResource: Plate actually inherits from ItemizedResource[Well], which means the type checker will raise an error if you initialize it with items other than a Well.
This is a form of documentation, loosely enforced by the type checker. Runtime type checking is not typical or recommended in Python, because it adds a lot of code (two lines per type check at least: if not isinstance(r, Well):\n raise error). Instead, the convention is to let it happen until some error occurs. If the user uses the API as documented (following type hints) we promise it will work and no error should occur (this is what the type checker enforces), otherwise the user is in control. We shouldn’t really care if the user wants to do experimental stuff.
And you’re right, we should document and formalize these rules more. Documentation has not always been a priority, but is becoming one now (I made a first step by redesigning the docs website for extensibility last week: new docs theme by rickwierenga · Pull Request #249 · PyLabRobot/pylabrobot · GitHub). Incidentally, Plate is one of the resources that has some documentation: Plates — PyLabRobot documentation, and we actually do say that the children will be Wells. But point taken.
The ‘strict rules’ as we will document them will simply be loosely enforced in the Python implementation, as is customary for Python implementations.
As for strictness checking in LiquidHandler, strictness checking makes the code more complicated. Because of it, we have to look at backend kwargs, do a set difference, potentially raise errors, etc. It only exists to facilitate flexible backends and help users write robot agnostic protocols.
Agree we need ‘unambiguous, well documented, fixed answers’.
The PLR resource model is still being developed, @CamilloMoschner has made great contributions here. The z coordinate of tip spots has not been iterated since my initial implementation*, and so I’m happy to discuss a new definition and changing it. Do you want to make a separate thread for it?
*my initial implementation attempted to stay close to venus. We are moving away from that because venus made some very bad decisions. Some things in PLR do not have good rationales behind right now.
Thank you for clearly stating this. Truly important feedback. We will improve.
It’s not. But why bother checking? No one will do it, so we just ignore it.
Yes
Speaking for Camillo, I think he is referring to “tip rack → tips pot → tip” is physical reality, and so is “plate → well”, but not "“plate → ‘well holder’ → well”. I would agree with Camillo that 2 or three layers stay closer to physical reality.
In the visualizer, I visualize tip spots and wells. Tip spots are separate objects (they have their own class), which change their appearance based on whether they have a tip assigned.
I don’t think it has to. But it does require using a tree as the fundamental data structure of the layout (which really is a core assumption) and nodes/classes in this tree to refine basic Resource behavior. Each node needs to handle their ‘local’ responsibility (eg Plates should manage returning Wells, but Plates should not manage the liquids in those wells (that is the job of Well))
Thanks a lot for the detailed responses, to both of you.
Our starting points for modelling data are different, which is a source of friction right now, but hopefully it means that there is something to gain on both ends.
I now understand a bit better what PLR is aiming at.
Perhaps the thread about Tips as Containers can host that discussion too.
