I long for a logging format in the specification. The requirements for a logging format:
Should we attempt to utilize ulog?
https://dev.px4.io/v1.9.0/en/log/ulog_file_format.html
If not, is there a better standards to adopt?
I think we should attempt to utilize what we have: the early-stage experimental PoC of the UAVCAN/serial transport. I am not sure if it’s entirely compatible with the requirement of being self-describing though.
I proposed earlier that we should log transport-layer data directly (like raw CAN frames) rather than processed higher-level entities (like DSDL objects) because it provides more context for later analysis and enables robust dumb loggers that do not understand UAVCAN and need not be configured. The idea is to construct DSDL objects of the appropriate type from uavcan.metatransport.*, then serialize them into UAVCAN/serial transfers, and store the result into a headerless binary file. This way we naturally get extensibility because we can also serialize arbitrary metadata, like uavcan.diagnostic.Record. The thing that I consider to be the most important here is that such logging solution is built upon things that are already available in UAVCAN; it’s self-sustaining.
I think we should postpone including things like this into the spec. We somewhat talked about this before:
Scott: should we define a UAVCAN storage format as part of the specification?
Pavel: I am not certain yet. UAVCAN is about real-time communication, not about data storage; the latter belongs to a different domain and does not affect many of the things that UAVCAN is concerned about, such as wire compatibility and functional correctness. At any rate, I am pretty sure it’s safe to just avoid answering this question for a while until we have gathered substantial empirical data on this feature.
We should revisit this question again if we decided to standardize the serial transport. In this case, the specification of UAVCAN/serial would seem to be the right place to document the log storage format.
I’m finding this question keeps coming up, “if UAVCAN messages are statically defined then how do I parse datasets that combine messages from multiple versions of my messages?” This is for use-cases where message data is aggregated across a fleet into a single system for analytics and/or prognostics. A way is needed to know the version of each message especially where incompatible versions of the messages were used on different vehicles. Given that we want UAVCAN to be “the batteries you really need are included” and that this is something almost any serious deployment of the protocol will need I think it’s time we talk about how to bring it in-scope. Whether this is a section of the core specification or perhaps a standalone specification we want an easy answer to this question.
What if we started a different specification or just ebook titled: “UAVCAN system management” that describes how you solve logging, bus-bandwidth calculations, deployments, health-monitoring, diagnostics, etc? All the stuff we keep saying is above the level of the core specification.
Great. I insist though that the picture of the cat be featured in the book. How do you want to approach it? Maybe we could post drafts right here on the forum and then at some point later assemble them into a more structured format? Or do you prefer a more fundamental approach as we did with the Specification?
Regarding the versioning issue, I think the data type hash as described in the post about alternative transports could solve this. Though the hash requires some modifications.
Sure, we can get started here. First, I think the datatype hash is something we should figure out and add to the core specification.
So is this a book on just system management or on design and management of UAVCAN systems?
Here’s a start at the TOC
It does seem very sensible and relevant.
Except for “Message Scheduling for complex busses”, of course. I would have declined to contribute only if it were about time-triggered communication, but message scheduling does not necessarily imply TT. Asynchronous message scheduling exists and it does not require TT:
But, what I meant when I said about posting here is that if we are ready to share a bit of UAVCAN wisdom with the world, we should make a forum thread or a dozen and let them sit for a while. Collect questions and criticism, etc. Kind of what I attempted to do here: Idempotent interfaces and deterministic data loss mitigation, or maybe even here: RTOS CAN API requirements. Then, having a critical mass of such items accumulated, we could squeeze them all into an e-book, slap a cat pic on top, and ship it 
.
I agree with all this. I want the first thread, this thread, to be the ToC. It’s actually an important list of “things we didn’t put in the core specification that you’ll probably care about if you build any real system using UAVCAN.” We should discuss, debate, and maintain this ToC breaking out individual topic as we have need and time.
