Yukon design megathread

[pavel.kirienko]

@adolfogc

Hi Pavel, I’m interested in helping with PyUAVCAN. I tried looking for a dev branch on PyUAVCAN’s GitHub repo but couldn’t find one. Could you point me to where this new version is being developed?

The devlopment is currently happening in my fork: https://github.com/pavel-kirienko/pyuavcan/tree/uavcan-v1.0. Seeing as you are interested, I should move it into the UAVCAN org. I am going to open tickets for coordination afterwards.

@Zarkopafilis

Would it not be easier to leave this to the user, at least for now? The user (being capable of abstract thinking and understanding, seeing as it’s likely to be a human being) should be able to deduce what’s happening by looking at the available controls & indicators.

That would do for now. I was thinking that we could maybe check for the first non-file request UAVCAN message with source ID equal to the one of the node that was previously updating, in order to determine online status.

Online nodes can be detected by listening for uavcan.node.Heartbeat – every UAVCAN node is required to publish this message at least once per second. When the uptime value reported by this message leaps backward, we know that the node has restarted. This, however, is unlikely to help with the software update tracking since a node does not necessarily have to restart to apply the updates.

@Zarkopafilis

That’s also needed, but I was asking for the more simple format, of copying a TypeValue . For now, the server returned type is copied to clipboard along with the _type_ , it could be handy to maybe export it in yaml or csv, or something else, idk, everything is possible.

You seem to be talking about a different feature. The format I described is supposed to store raw data frames without any higher-level data whatsoever. This is what I am suggesting for use for data log storage. If it ended up being accepted, we will need to eventually teach Yukon how to open such data logs and extract application-level information out of them. For now this is not yet critical.

@Zarkopafilis

It seems like that on the gui_tool, only CAN frames are shown, I am thinking of a design that could show all the frame types, without that being visually noisy. Do you have any recommendations for that?

I do. Suppose there are the following columns in the log:

1. monotonic timestamp
2. UTC or local timestamp (see below)
3. transport frame header
4. priority level name per the spec (exceptional, immediate, fast, high, nominal, low, slow, optional)
5. route specifier:
   • source node ID
   • destination node ID (empty for broadcast)
6. data specifier:
   • if message: subject ID
   • if service call: service ID and request/response selector
   • data type name and version (the mapping from subject/service ID is to be defined by the user or auto-deduced by parsing the logs)
7. transfer ID
8. data in hex
9. data in ASCII

If you are curious where did “route specifier” and “data specifier” come from, read this: Alternative transport protocols in UAVCAN, there is a diagram.

Obviously, UAVCAN may share the same medium/transport with other protocols, so it is expected that for some frames we will not be able to determine “route specifier” and “data specifier” because they have nothing to do with UAVCAN, so the respective columns will be left unpopulated.

Now, the column “transport frame header” is supposed to contain the low-level transport frame information in the raw form, completely unprocessed. Hence, its contents will be dependent on the kind of the transport protocol in use. Per the table I linked from my previous post, the values would be at least the following:

• For CAN bus:
  • CAN ID
  • flags: extended or base frame, RTR frame, error frame, CAN FD frame
• For UDP/IP (lots of data here):
  • source endpoint (IPv6/v4 address, MAC address, port)
  • destination endpoint
• For IEEE 802.15.4:
  • source MAC address (64 bit)
  • destination MAC address
  • source short address (16 bit)
  • destination short address
  • RSSI indicator

For consistency, I suppose all that data should be squeezed into one column.

UTC/local timestamps are tricky. If you look closely at my proposed data log format, you will see that it only mentions a “monotonic timestamp”, which has no well-defined starting point and its only purpose is measuring time intervals between messages belonging to the same recording session (where recording sessions are separated with special records where the transport ID is zero). It is important to stick to monotonic because globally managed time such as UTC or GPS is not always available, yet we don’t want to pause logging if the time information turned out to be missing; additionally, such synchronized clocks may change rate or leap in order to stay synchronized, which is disastrous for logging. So monotonic time is easy to record, but how do we convert that to a more usable global time? I suggest we extract the necessary information directly from the logged data. If your system synchronizes its time with UTC or GPS, this time information will eventually occur in the dump. We scan the dump looking for time synchronization messages. Having found one, we take the time and subtract the monotonic timestamp from it; then this difference can be applied across the whole log to determine the UTC/GPS time of any other logged frame. Additionally, as a fallback option the system that wrote the log file may inject additional UTC/GPS/whatever time records into the recorded log file by using one of the reserved metadata entry formats, so that if the logged system turned out to exchange no time-related information, it could still be extracted from these additional metadata entries. The worst special case is when the data log contains neither logged time information nor time metadata entries, in which case the corresponding time column should remain unpopulated unless the user enters the time difference manually.