I’m thinking more about how to better test our Cyphal networks and curious if anyone has example setups for how they do HITL testing, fault injection, or fuzz their Cyphal networks. Is there a framework that people have used for this kind of work and have had good experiences with? Is this a broader project that the consortium would be interested in exploring more? I think having an established set of testing tools would be a good way to further encourage adoption, and be good for the standard and community as well.
This is a very interesting topic but I’m afraid I don’t have any generalized answers. Compatibility (happy case) is often tested using pycyphal interoperability as an indication that the device is standards compliant but we don’t have a standard suite of interactions that would constitute full compatibility. I think this would be the first step, to automate a pycyphal session (possibly via yakut) to cover all required functionality. After that, fault injection could be added by modifying pycyphal to add error injection hooks.
Contributions would be welcome for this type of test suite.
For the past year and a half I’ve been working on FluxGrip, which has among others a bunch of verification tests for testing the Cyphal-related functionality. (Turning the magnet on/off by sending a uavcan.primitive.scalar.Integer8, checking whether the on/off counter gets updated in zubax.fluxgrip.Feedback message,…)
I have written a forum post on how I did the setup for that project here. The code can be found here.
The repo does not contain any of my Cyphal-related integration tests (in my private FluxGrip repo they can be found in verification/integration/tests/test_0001_pnp.py, verification/integration/tests/test_0002_magnet.py,…), but I suppose that once libcyphal is done it would be a useful exercise to update that repo to also have some basic example on how to test Cyphal functionality.
(Now that I think of it, this feels like a perfect task for an intern. @pavel.kirienko)