How to avoid excessive memory usage?

Я начал реализовывать Cyphal на своих устройствах и столкнулся с невероятной расточительностью по памяти. Все вроде бы шло хорошо, но я дошел до реализации uavcan.node.port.List и ужаснулся. Допустим мое устройство подписано на пару топиков и парочку отправляет. Итого , в принципе мне надо держать не так много данных в памяти , весь список. Но вместо пары десятков байт реальных данных я должен держать структуру размером 8466 байт , допустим, это можно порешать , ок , но … после сериализации я получаю на выходе 2194 сериализованных данных( кстати, почему это число такого порядка я так и не понял ) … ну , хорошо … Самое противное что я должен отправить порядка 250 Can2.0 фреймов … чтобы передать , жалких 30 байт полезной информации … . Это все как то не очень ни с точки зрения памяти , ни с точки зрения здравого смысла. Есть какой то способ как это можно пофиксить ??? Yukon , как я понял, без этого сообщения ничего нормально отображать не будет , а очень хотелось им бы пользоваться.

It is best to stick to English because few people on this forum can read Russian.

When it comes to embedded systems, Cyphal/CAN is mostly intended for use on platforms that have at least 32 KiB of RAM (Cyphal/UDP is expected to require at least 64 KiB), but even then, you may still need to jump through some hoops to keep the memory usage in check, depending on the needs of the application.

The uavcan.node.port.List is indeed a very large message because it has to carry a large amount of information about the publishing node — specifically, the full set of all active publishers/subscribers/clients/servers; given this, it is hard to come up with a substantially more compact alternative (unless you’re willing to embrace statefulness, which we’re not).

If your application cannot spare 8K for this, I recommend resorting to manual serialization: just allocate a raw memory buffer large enough to hold your serialized message (which is going to be less than 200 bytes) and then encode each field one by one following the serialization rules defined in the Specification (you can also consult with the autogenerated code). The serialization primitives from the Nunavut support library (nunavutSetUxx et al.) are going to be of great help here.

The size of the serialization buffer is derived from the worst-case (maximum) size of all nested types. If you only need to report a few ports, the actual size is going to be much lower, which is why you only need <200 bytes. Observe: the uavcan.node.port.List message contains four nested fields:

SubjectIDList.1.0 publishers
SubjectIDList.1.0 subscribers
ServiceIDList.1.0 clients
ServiceIDList.1.0 servers

The first two are unions; you need this option that requires two bytes per port plus one byte for the array length prefix and one byte for the union tag:

SubjectID.1.0[<256] sparse_list

So if you have x publishers and y subscribers, you need 4+2x+2y bytes. Then for services we have two fields of type ServiceIDList which is a fixed-size type, 64 bytes each. All of the types are non-sealed, meaning that each requires a four-byte delimiter header. Therefore, the actual size of the message is 4+2x+2y+2\times{}64+4\times{}4.

I’ve been thinking that for messages like this one we could probably generate alternative serialization functions that accept fields as arguments instead of a fully-populated message object, as it would help reduce memory usage. Would you be willing to collaborate on that?

Also, another interesting thing to work on is some kind of interactive deserializer that allows you to paste a binary blob in one input field, a DSDL source text in another input field, and then see how the contents of the blob map onto the DSDL source interactively. Kinda like but for DSDL.

1 Like

Thanks for the reply. Indeed, the length of the serialized buffer returned to normal after I set “publishers.tag =1”. And the calculated length became really equal to the given formula

1 Like