# Adding io layer

**URL:** <https://discuss.pylabrobot.org/t/adding-io-layer/149>\
**Category:** General\
**Created:** [February 5, 2025, 3:12am UTC](https://discuss.pylabrobot.org/t/adding-io-layer/149 "2025-02-05T03:12:46Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rickwierenga](https://discuss.pylabrobot.org/user_avatar/discuss.pylabrobot.org/rickwierenga/32/963_2.png) [@rickwierenga](https://discuss.pylabrobot.org/u/rickwierenga)\
**Post date:** [February 5, 2025, 3:12am UTC](https://discuss.pylabrobot.org/t/adding-io-layer/149/1 "2025-02-05T03:12:46Z")

</div>

[This PR](https://github.com/PyLabRobot/pylabrobot/pull/392) introduces a third layer of abstraction in PLR: the io layer. This will live in between backends and the physical reality. ALL communication of PLR with machines will go through this layer. Some IO objects will be thin wrappers, others will be more complicated.

Backends will now have an `io` attribute, similar to have frontends have a backend attribute. Backends will still be responsible for creating machine specific commands, but communication will be done in the io layer.

 ![](https://discuss.pylabrobot.org/uploads/default/original/1X/98b218c73c7062d7c8ed2f67f37ea239b8cf14ea.png)

Why? This is for testing. If the communication that goes through the io layer doesn’t change, it means we are communicating the same commands to the outside world: machines will do the same thing. Having ALL communication in this layer makes that easy. It is now easy to write integration tests. When you change something in your protocol or pull from plr:main, if the io layer sends and receives the same data, the protocol will be the same.

The source of truth is a log file generated during a real automation run (or a liquid test). This means that the protocol is _proven_ to work in physical reality, otherwise you wouldn’t get a log file (tempering is possible, but you’re your own adversary). When you change your protocol, you can easily grab that new log file and make it the file that you validate against. No manually updating tests.

Checking happens through a central object called the LogReader. This reads the commands in a log file. The IO objects will then validate that if they receive a `write` command the data matches that in the log file. If they receive a read command they will check that in physical reality, the specific device actually returned some data (the validation object will then actually return this data so that the backend can function normally. The objects in the IO layer (USB, FTDI, Serial, …) will all call into the single LogReader and request the next line. This ensures that not only do the IO objects read/write the correct information, they also do so in the right order. For example, the door on your cytomat is opened _before_ the iswap puts the plate in, not after.

An added benefit of checking against log files is that more complicates sequences like [`step_off_foil`](https://github.com/PyLabRobot/pylabrobot/pull/357) will work even in testing. `step_off_foil` requests the channel positions and positions the channels based on that. Previously, it was impossible to test this function.

You can find an example notebook in [`docs/user_guide/validation.ipynb`](https://github.com/PyLabRobot/pylabrobot/blob/8ddb028dacc466f5e8829eeaeef7a46284a6c30e/docs/user_guide/validation.ipynb)

> <https://github.com/PyLabRobot/pylabrobot/pull/392>
>
> This PR introduces a third layer of abstraction in PLR: the io layer. This will …live in between backends and the physical reality. ALL communication of PLR with machines will go through this layer. Some IO objects will be thin wrappers, others will be more complicated.
> 
> Backends will now have an \`io\` attribute, similar to have frontends have a backend attribute. Backends will still be responsible for creating machine specific commands, but communication will be done in the io layer.
> 
> !\[\](https://discuss.pylabrobot.org/uploads/default/original/1X/98b218c73c7062d7c8ed2f67f37ea239b8cf14ea.png)
> 
> Why? This is for testing. If the communication that goes through the io layer doesn't change, it means we are communicating the same commands to the outside world: machines will do the same thing. Having ALL communication in this layer makes that easy. It is now easy to write integration tests. When you change something in your protocol or pull from plr:main, if the io layer sends and receives the same data, the protocol will be the same.
> 
> The source of truth is a log file generated during a real automation run (or a liquid test). This means that the protocol is \_proven\_ to work in physical reality, otherwise you wouldn't get a log file (tempering is possible, but you're your own adversary). When you change your protocol, you can easily grab that new log file and make it the file that you validate against. No manually updating tests.
> 
> Checking happens through a central object called the LogReader. This reads the commands in a log file. The IO objects will then validate that if they receive a \`write\` command the data matches that in the log file. If they receive a read command they will check that in physical reality, the specific device actually returned some data (the validation object will then actually return this data so that the backend can function normally. The objects in the IO layer (USB, FTDI, Serial, ...) will all call into the single LogReader and request the next line. This ensures that not only do the IO objects read/write the correct information, they also do so in the right order. For example, the door on your cytomat is opened \_before\_ the iswap puts the plate in, not after.
> 
> An added benefit of checking against log files is that more complicates sequences like \[\`step\_off\_foil\`\](https://github.com/PyLabRobot/pylabrobot/pull/357) will work even in testing. \`step\_off\_foil\` requests the channel positions and positions the channels based on that. Previously, it was impossible to test this function.
> 
> You can find an example notebook in \`docs/user\_guide/validation.ipynb\`

---

<div class="post-metadata">

**Author:** ![rickwierenga](https://discuss.pylabrobot.org/user_avatar/discuss.pylabrobot.org/rickwierenga/32/963_2.png) [@rickwierenga](https://discuss.pylabrobot.org/u/rickwierenga)\
**Post date:** [February 5, 2025, 3:16am UTC](https://discuss.pylabrobot.org/t/adding-io-layer/149/2 "2025-02-05T03:16:00Z")

</div>

it actually uses Needleman Wunsch to align firmware commands in case of an error!

 ![image](https://discuss.pylabrobot.org/uploads/default/original/1X/61280ec11be55d1d13120d7d56bda2847b477728.png)l

---

<div class="post-metadata">

**Author:** ![koeng](https://discuss.pylabrobot.org/letter_avatar_proxy/v4/letter/k/4af34b/32.png) [@koeng](https://discuss.pylabrobot.org/u/koeng)\
**Post date:** [February 5, 2025, 6:46pm UTC](https://discuss.pylabrobot.org/t/adding-io-layer/149/3 "2025-02-05T18:46:26Z")

</div>

Would this be something we want to implement in the opentrons backend? Ie, intercepting some kind of log files to check the expected values. We could do this either by gathering the log files from opentrons\_simulate, or by making our own fake io LogReader with more consistent nomenclature.

The disadvantage of `opentrons_simulate` is the output changes per opentrons version. The disadvantage of our own stack is that the `io` might not guarantee changes are consistent, as we play around with getting the interface right.

---

<div class="post-metadata">

**Author:** ![rickwierenga](https://discuss.pylabrobot.org/user_avatar/discuss.pylabrobot.org/rickwierenga/32/963_2.png) [@rickwierenga](https://discuss.pylabrobot.org/u/rickwierenga)\
**Post date:** [February 5, 2025, 7:10pm UTC](https://discuss.pylabrobot.org/t/adding-io-layer/149/4 "2025-02-05T19:10:40Z")

</div>

yes, since it will use the HTTP object in the io layer (thin wrapper around python `requests`). It is testing HTTP requests to your server
