We’re a student team from Revolve NTNU, working on a new telemetry system using Cyphal. Our data is sent over UDP, and we’re thinking about using PCAN for USB and PCAP for our power unit.
We already have a .NET setup, but we’re considering switching to Telegraf.
Our team consists of four students with diverse Computer Science backgrounds. We’re new to Cyphal and eager to learn.
Our aim is to feed our data into Telegraf. We’re looking into making custom plugins to process and convert Cyphal data into Telegraf Metrics. Our initial idea is to use Telegraf’s UDP plugin and add a custom Cyphal parser.
We’ve been checking out the go-canard project since Telegraf plugins use Go.
Any advice or tips from the community would be super helpful!
What is the reason to use Cyphal for telemetry?
If telemetry means in your context collecting data from sensors that are connected to a network - then use MQTT with Telegraf support out of the box.
The entire car is a CAN network. Yes the VCU could somehow connect to the MQTT broker and produce MQTT messages but software wise this only adds complexity to the car side, which is something we are trying to avoid.
This is closely related to something we’ve been considering for a while. The need to collect operational data from a vehicle and then upload it somewhere offboard for long-term aggregation and eventual processing (incl. data mining when larger-scale operations are involved) is quite common in the industry involving vehicles of many kinds, from agricultural drones and racing cars up to airliners. At least in the applications we’re involved with it is desirable to retain the original low-level syntax and appearance of the data as much as possible, as this extended context may eventually prove useful when said data is being analyzed or mined at a later stage; practically this means that in an idealized scenario, one should aim to capture all network traffic from the vehicle verbatim and store it in the form of raw captured network packets (e.g., raw CAN or Ethernet frames as-is as opposed to some higher-level message representations) with accurate timestamps.
At least the first approximation of a solution is a hardware unit permanently installed onboard that logs data into a non-volatile buffer that is large enough to last at least for the entire mission. The unit should be equipped with some form of wireless connectivity that, upon connection to the Internet (e.g., while the vehicle is being serviced), would automatically upload the collected data to a server starting from the least recent records. Uploaded records would be removed from the onboard unit, thus making it a zero-maintenance solution that continuously keeps the data stored on the server up to date, provided that there is at least intermittent Internet connectivity available. As long as the backlog of the records stored on the onboard unit is empty and there is Internet connectivity available, network frames appearing in the onboard bus will be immediately mirrored on the server in soft real-time, enabling real-time diagnostics.
For high-traffic topics, decimation may be required to manage the onboard storage requirements and the log synchronization time, especially if the (intermittent) Internet connectivity is bandwidth-limited.
One might see how this naturally extends toward a slightly different, albeit related diagnostic application: if the system were to support reverse data transfer from the server to the vehicle, it could be leveraged to actively probe and control the system remotely as long as it is connected to the Internet, thus obviating the need for dedicated interface adapters (such as USB-CAN bridges or network cards for Cyphal/UDP). Then, the next obvious step is to extend our standard Cyphal diagnostic tools — Yakut, Yukon, and PyCyphal — such that one could tap into the network of any vehicle in any given fleet and exchange data with that vehicle in real-time (if there is Internet connectivity) or replay previously recorded data.
For the Yukon GUI tool, this is particularly interesting as it enables the user to interact with the service from the web browser directly without the need to install any software locally or interact with the local hardware in any way.
This proposal includes a wide range of solutions in it. There are, perhaps, parts of it I find interesting and other parts I do not. If you were to propose two, separable-if-compatible solutions; one for vehicle to cloud and another for cloud to diagnostic tools I would be interested in the latter.
Interesting topic, and one that deserves more attention I believe.
Very true, you need to go down to the bits to be able to properly troubleshoot things. Only thing I would add is that metadata should also be added (ex: timestamps).
This is something that I think will eventually become the de-facto, same a flight recorders on planes. Only thing I dislike about the approach you proposed is that it’s only effective after the fact, not during. Imagine your car gauges showing you what happened in the past…
I don’t think there’s a golden egg for this, I think this can be split into two problems, each requiring a different solution:
1: Black Box (aka Flight Recorder): What you proposed sounds perfect. The data could also be used for things like simulations…
2: Real Time Analytics: I think a different approach is needed for this. In the case of CAN, I’m imagining a Cyphal/CAN to Cyphal/UDP bridge where the CAN packets are wrapped up along with metadata and forwarded via UDP. I could also imagine other messages with information such a bus load.
Reality is, the cost of debugging, testing, and verification is estimated to account for 50-75 percent of the total budget of software development projects. If you’re shooting in the dark, that percentage will surely be way higher.
