# How to avoid excessive memory usage?

**URL:** <https://forum.opencyphal.org/t/how-to-avoid-excessive-memory-usage/2083>\
**Category:** Libcanard\
**Created:** [February 16, 2024, 2:12pm UTC](https://forum.opencyphal.org/t/how-to-avoid-excessive-memory-usage/2083 "2024-02-16T14:12:14Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ozoni](https://forum.opencyphal.org/letter_avatar_proxy/v4/letter/o/b3f665/32.png) [@Ozoni](https://forum.opencyphal.org/u/Ozoni)\
**Post date:** [February 16, 2024, 2:12pm UTC](https://forum.opencyphal.org/t/how-to-avoid-excessive-memory-usage/2083/1 "2024-02-16T14:12:14Z")

</div>

Я начал реализовывать Cyphal на своих устройствах и столкнулся с невероятной расточительностью по памяти. Все вроде бы шло хорошо, но я дошел до реализации uavcan.node.port.List и ужаснулся. Допустим мое устройство подписано на пару топиков и парочку отправляет. Итого , в принципе мне надо держать не так много данных в памяти , весь список. Но вместо пары десятков байт реальных данных я должен держать структуру размером 8466 байт , допустим, это можно порешать , ок , но … после сериализации я получаю на выходе 2194 сериализованных данных( кстати, почему это число такого порядка я так и не понял ) … ну , хорошо … Самое противное что я должен отправить порядка 250 Can2.0 фреймов … чтобы передать , жалких 30 байт полезной информации … . Это все как то не очень ни с точки зрения памяти , ни с точки зрения здравого смысла. Есть какой то способ как это можно пофиксить ??? Yukon , как я понял, без этого сообщения ничего нормально отображать не будет , а очень хотелось им бы пользоваться.

---

<div class="post-metadata">

**Author:** ![pavel.kirienko](https://forum.opencyphal.org/user_avatar/forum.opencyphal.org/pavel.kirienko/32/9_2.png) [@pavel.kirienko](https://forum.opencyphal.org/u/pavel.kirienko)\
**Post date:** [February 17, 2024, 5:32pm UTC](https://forum.opencyphal.org/t/how-to-avoid-excessive-memory-usage/2083/2 "2024-02-17T17:32:25Z")

</div>

It is best to stick to English because few people on this forum can read Russian.

When it comes to embedded systems, Cyphal/CAN is mostly intended for use on platforms that have at least 32 KiB of RAM (Cyphal/UDP is expected to require at least 64 KiB), but even then, you may still need to jump through some hoops to keep the memory usage in check, depending on the needs of the application.

The `uavcan.node.port.List` is indeed a very large message because it has to carry a large amount of information about the publishing node — specifically, the full set of all active publishers/subscribers/clients/servers; given this, it is hard to come up with a substantially more compact alternative (unless you’re willing to embrace statefulness, which we’re not).

> [@Ozoni](#):
>
> вместо пары десятков байт реальных данных я должен держать структуру размером 8466 байт

If your application cannot spare 8K for this, I recommend resorting to manual serialization: just allocate a raw memory buffer large enough to hold your serialized message (which is going to be less than 200 bytes) and then encode each field one by one following the serialization rules defined in the Specification (you can also consult with the autogenerated code). The serialization primitives from the Nunavut support library (`nunavutSetUxx` et al.) are going to be of great help here.

> [@Ozoni](#):
>
> после сериализации я получаю на выходе 2194 сериализованных данных( кстати, почему это число такого порядка я так и не понял ) … ну , хорошо … Самое противное что я должен отправить порядка 250 Can2.0 фреймов … чтобы передать , жалких 30 байт полезной информации …

The size of the serialization buffer is derived from the worst-case (maximum) size of all nested types. If you only need to report a few ports, the actual size is going to be much lower, which is why you only need \<200 bytes. Observe: the `uavcan.node.port.List` message contains four nested fields:

```python
SubjectIDList.1.0 publishers
SubjectIDList.1.0 subscribers
ServiceIDList.1.0 clients
ServiceIDList.1.0 servers

```

The first two are unions; you need this option that requires two bytes per port plus one byte for the array length prefix and one byte for the union tag:

```python
SubjectID.1.0[<256] sparse_list

```

So if you have x publishers and y subscribers, you need 4+2x+2y bytes. Then for services we have two fields of type `ServiceIDList` which is a fixed-size type, 64 bytes each. All of the types are non-`sealed`, meaning that each requires a four-byte delimiter header. Therefore, the actual size of the message is 4+2x+2y+2\times{}64+4\times{}4.

I’ve been thinking that for messages like this one we could probably generate alternative serialization functions that accept fields as arguments instead of a fully-populated message object, as it would help reduce memory usage. Would you be willing to collaborate on that?

Also, another interesting thing to work on is some kind of interactive deserializer that allows you to paste a binary blob in one input field, a DSDL source text in another input field, and then see how the contents of the blob map onto the DSDL source interactively. Kinda like [godbolt.org](http://godbolt.org) but for DSDL.

---

<div class="post-metadata">

**Author:** ![Ozoni](https://forum.opencyphal.org/letter_avatar_proxy/v4/letter/o/b3f665/32.png) [@Ozoni](https://forum.opencyphal.org/u/Ozoni)\
**Post date:** [February 26, 2024, 7:19am UTC](https://forum.opencyphal.org/t/how-to-avoid-excessive-memory-usage/2083/3 "2024-02-26T07:19:16Z")

</div>

Thanks for the reply. Indeed, the length of the serialized buffer returned to normal after I set “publishers._tag_ =1”. And the calculated length became really equal to the given formula
