"How do you test whether machines perform the same as the manufacturers software?"

General
1

core idea: if you send the same bytes over the wire, the machine will do the same thing

2 Likes
2

Confirm the intended transfer pattern, then confirm the volumes using an Artel. IRL unit testing is so time consuming

3

you don’t even really need to test in physical reality after you know communications works because how the machine responds to firmware commands is their responsibility. getting communication working can be a challenge, and it’s always good to sanity check, but for example our unit tests don’t require machines because we trust that sending identical firmware commands will lead to identical behavior.

4

The challange lies im the definition of what means “the machines do the same”. You are correct, sending the same commands as vendor SW will lead to the same action / behaviour of the machine. But on a higher level it’s hard to get PLR to send the same commands: different labware definitions, liquid classes, … all will affect the commands / parameters sent to the machine.

But do you even want PLR to send exactly the same commands? In the end you want the correct results, no matter which SW was used.

So it depends on what you verify:

  • Verifiying if two versions of PLR do the same? → compare what those two version send to the (simulated) device. If it’s the same, the result will be the same.
  • Verify if a PLR script run does the same as a vendor SW run? → check the volumes / compare the biological results. Or, if you have a good understanding of the FW, compare the commands sent to the device and figure out if the differences are relevant for the outcome.
2 Likes