# Proposed change to variable-length arrays

**URL:** <https://forum.opencyphal.org/t/proposed-change-to-variable-length-arrays/740>\
**Category:** Specification\
**Created:** [January 31, 2020, 4:53pm UTC](https://forum.opencyphal.org/t/proposed-change-to-variable-length-arrays/740 "2020-01-31T16:53:49Z")\
**Posts on this page:** 1\
**Showing post:** 7

<div class="post-metadata">

**Author:** ![pavel.kirienko](https://forum.opencyphal.org/user_avatar/forum.opencyphal.org/pavel.kirienko/32/9_2.png) [@pavel.kirienko](https://forum.opencyphal.org/u/pavel.kirienko)\
**Post date:** [March 6, 2020, 4:08pm UTC](https://forum.opencyphal.org/t/proposed-change-to-variable-length-arrays/740/7 "2020-03-06T16:08:23Z")

</div>

> [@pavel.kirienko](#):
>
> I think that explicit alignment with `@align` is more _ **aligned** _ 🥁 with the core design goals, but this is probably debatable.

We need to revisit this now because after we [accepted the bit order change](https://forum.opencyphal.org/t/big-endian-vs-little-endian-in-the-context-of-bit-level-encoding/428/27) some of the issues involved in this discussion have been altered. I mentioned this issue briefly in the linked topic, too.

Imagine we have an array type like this:

```python
void7
uint8[<=256] a

```

One might expect that its serialized representation would look like the length field promoted to uint16 encoded in the little-endian byte order and followed by the array data, for example:

```python
>>> [bytes(x) for x in pyuavcan.dsdl.serialize(uavcan.primitive.String_1_0(
... 'Yesterday I yelled a degrading insult at an elderly lady.'))]
[b'\x39\x00Yesterday I yelled a degrading insult at an elderly lady.']

```

Where `\x39\x00` would be the length of the string in the little-endian byte order. That would be intuitive and extensible since it would be possible to chop off a bit from the padding to enlarge the capacity. What we get instead with the new bit ordering rule is:

```python
>>> [bytes(x) for x in pyuavcan.dsdl.serialize(uavcan.primitive.String_1_0(
... 'Yesterday I yelled a degrading insult at an elderly lady.'))]
[b'\x80\x1cYesterday I yelled a degrading insult at an elderly lady.']

```

Where `\x80\x1c` is the least significant bit of the length shifted up seven places followed by `57 >> 1`.

Perhaps, the resulting bit layouts may be considered to contradict the core principles by being more convoluted than one might think by looking at a DSDL definition? In that view, I think your proposal to make implicit fields byte-size-aligned should probably be accepted by changing the following two sections of the Specification:

- 3.7.4.2 Variable-length array type
- 3.7.5.1 Tagged unions

Requiring the implementation to round the size of an implicit field up to the smallest of `[8, 16, 32, 64]`.

A directive reversing this behavior can be introduced as well later.

_ **Edit:** Fields from the regulated set that will be affected, including the preceding manual padding:_

```plaintext
$ grep '\[<' -R . -B1
./uavcan/diagnostic/32760.Record.1.0.uavcan-void6
./uavcan/diagnostic/32760.Record.1.0.uavcan:uint8[<=112] text
--
./uavcan/file/408.Read.1.0.uavcan-void7
./uavcan/file/408.Read.1.0.uavcan:uint8[<=256] data
--
./uavcan/file/409.Write.1.0.uavcan-
./uavcan/file/409.Write.1.0.uavcan:uint8[<=192] data # 192 = 128 + 64; the write protocol permits usage of smaller chunks.
--
./uavcan/file/Path.1.0.uavcan-void1
./uavcan/file/Path.1.0.uavcan:uint8[<=MAX_LENGTH] path
--
./uavcan/internet/udp/500.HandleIncomingPacket.0.1.uavcan-void7
./uavcan/internet/udp/500.HandleIncomingPacket.0.1.uavcan:uint8[<=309] payload
--
./uavcan/internet/udp/32750.OutgoingPacket.0.1.uavcan-void2
./uavcan/internet/udp/32750.OutgoingPacket.0.1.uavcan:uint8[<=45] destination_address
--
./uavcan/internet/udp/32750.OutgoingPacket.0.1.uavcan-
./uavcan/internet/udp/32750.OutgoingPacket.0.1.uavcan:uint8[<=261] payload
--
./uavcan/metatransport/can/DataClassic.0.1.uavcan-void4
./uavcan/metatransport/can/DataClassic.0.1.uavcan:uint8[<=8] data
--
./uavcan/metatransport/can/DataFD.0.1.uavcan-void1
./uavcan/metatransport/can/DataFD.0.1.uavcan:uint8[<=64] data
--
./uavcan/metatransport/serial/Fragment.0.1.uavcan-void7
./uavcan/metatransport/serial/Fragment.0.1.uavcan:uint8[<=CAPACITY_BYTES] data
--
./uavcan/metatransport/udp/Frame.0.1.uavcan-void2
./uavcan/metatransport/udp/Frame.0.1.uavcan:uint8[<=MTU] data
--
./uavcan/node/port/431.List.0.1.uavcan-
./uavcan/node/port/431.List.0.1.uavcan:uint16[<=128] port_ids # See above for the full description of the logic.
--
./uavcan/node/port/432.GetInfo.0.1.uavcan-void2
./uavcan/node/port/432.GetInfo.0.1.uavcan:uint8[<=50] data_type_full_name
--
./uavcan/node/434.GetTransportStatistics.0.1.uavcan-void6
./uavcan/node/434.GetTransportStatistics.0.1.uavcan:IOStatistics.0.1[<=MAX_NETWORK_INTERFACES] network_interface_statistics
--
./uavcan/node/430.GetInfo.1.0.uavcan-void2
./uavcan/node/430.GetInfo.1.0.uavcan:uint8[<=50] name
--
./uavcan/node/430.GetInfo.1.0.uavcan-void7
./uavcan/node/430.GetInfo.1.0.uavcan:uint64[<=1] software_image_crc
--
./uavcan/node/430.GetInfo.1.0.uavcan-
./uavcan/node/430.GetInfo.1.0.uavcan:uint8[<=222] certificate_of_authenticity
--
./uavcan/node/435.ExecuteCommand.1.0.uavcan-void1
./uavcan/node/435.ExecuteCommand.1.0.uavcan:uint8[<=uavcan.file.Path.1.0.MAX_LENGTH] parameter
--
./uavcan/pnp/cluster/32740.Discovery.1.0.uavcan-void2
./uavcan/pnp/cluster/32740.Discovery.1.0.uavcan:uavcan.node.ID.1.0[<=5] known_nodes
--
./uavcan/pnp/cluster/390.AppendEntries.1.0.uavcan-void7
./uavcan/pnp/cluster/390.AppendEntries.1.0.uavcan:Entry.1.0[<=1] entries
--
./uavcan/pnp/32742.NodeIDAllocationDataMTU8.0.1.uavcan-
./uavcan/pnp/32742.NodeIDAllocationDataMTU8.0.1.uavcan:uavcan.node.ID.1.0[<=1] allocated_node_id
--
./uavcan/primitive/array/Bit.1.0.uavcan-void4
./uavcan/primitive/array/Bit.1.0.uavcan:bool[<=2048] value
--
./uavcan/primitive/array/Integer16.1.0.uavcan:int16[<=128] value
--
./uavcan/primitive/array/Integer32.1.0.uavcan-void1
./uavcan/primitive/array/Integer32.1.0.uavcan:int32[<=64] value
--
./uavcan/primitive/array/Integer64.1.0.uavcan-void2
./uavcan/primitive/array/Integer64.1.0.uavcan:int64[<=32] value
--
./uavcan/primitive/array/Integer8.1.0.uavcan-void7
./uavcan/primitive/array/Integer8.1.0.uavcan:int8[<=256] value
--
./uavcan/primitive/array/Natural16.1.0.uavcan:uint16[<=128] value
--
./uavcan/primitive/array/Natural32.1.0.uavcan-void1
./uavcan/primitive/array/Natural32.1.0.uavcan:uint32[<=64] value
--
./uavcan/primitive/array/Natural64.1.0.uavcan-void2
./uavcan/primitive/array/Natural64.1.0.uavcan:uint64[<=32] value
--
./uavcan/primitive/array/Natural8.1.0.uavcan-void7
./uavcan/primitive/array/Natural8.1.0.uavcan:uint8[<=256] value
--
./uavcan/primitive/array/Real16.1.0.uavcan:float16[<=128] value # Exactly representable integers: [-2048, +2048]
--
./uavcan/primitive/array/Real32.1.0.uavcan-void1
./uavcan/primitive/array/Real32.1.0.uavcan:float32[<=64] value # Exactly representable integers: [-16777216, +16777216]
--
./uavcan/primitive/array/Real64.1.0.uavcan-void2
./uavcan/primitive/array/Real64.1.0.uavcan:float64[<=32] value # Exactly representable integers: [-2 **53, +2** 53]
--
./uavcan/primitive/String.1.0.uavcan-void7
./uavcan/primitive/String.1.0.uavcan:uint8[<=256] value
--
./uavcan/primitive/Unstructured.1.0.uavcan-void7
./uavcan/primitive/Unstructured.1.0.uavcan:uint8[<=256] value
--
./uavcan/register/Name.1.0.uavcan-void2
./uavcan/register/Name.1.0.uavcan:uint8[<=50] name
--
./regulated/zubax/actuator/esc/AngVelSetpoint.0.1.uavcan-void2
./regulated/zubax/actuator/esc/AngVelSetpoint.0.1.uavcan:uavcan.si.unit.angular_velocity.Scalar.1.0[<64] motor_mechanical_angular_velocity
--
./regulated/zubax/actuator/esc/RatiometricSetpoint.0.1.uavcan-
./regulated/zubax/actuator/esc/RatiometricSetpoint.0.1.uavcan:int10[<64] ratiometric
--
./regulated/zubax/sensor/bms/BatteryPackParams.0.1.uavcan-void2
./regulated/zubax/sensor/bms/BatteryPackParams.0.1.uavcan:uint8[<64] name
--
./regulated/zubax/sensor/bms/BatteryPackStatus.0.1.uavcan-
./regulated/zubax/sensor/bms/BatteryPackStatus.0.1.uavcan:float16[<256] cell_voltages # [volt]

```

---

_[View the full topic](https://forum.opencyphal.org/t/proposed-change-to-variable-length-arrays/740)._
