RFC: Zero-configuration Cyphal with named topics

The consensus algorithm relies on two CRDTs.

Topic age counter

This is needed to support an essential condition that newcomers do not disturb an existing network configuration. Without this provision, the network would likely be useless as any new topic or node could introduce a disturbance, potentially causing data loss. The topic age counter is incremented at some rate by all nodes that are members of that topic (meaning that they subscribe and/or publish on it, or at least are interested in potentially doing so at some point in the future). Every time a node broadcasts the metadata of that topic, the local age counter value is included, which allows all other nodes that are interested in this topic to synchronize their counters by simply choosing max(local_counter, remote_counter). The max function trivially satisfies the CRDT requirements as it is commutative, associative, and idempotent.

The age counter may differ by a few counts between participants because the counting never stops, while CRDT only guarantee eventual consistency. As elaborated in the README, nodes use \lfloor \log_2{\text{counter}} \rfloor for ranking topics to eliminate jitter.

Eviction counter (aka local collision Lamport clock)

When a node registers a new topic locally upon request from the application, it doesn’t yet know if this topic is already known to the network, and how it is allocated, so it starts from the default topic configuration: age zero, eviction count zero. The subject-ID of a topic is computed simply as (rapidhash(topic_name) + eviction_counter) % 6144, so the initial subject-ID is just the hash modulo subject-ID space. Subsequently, the topic metadata is gossiped to the network.

In a sparsely populated network, it will likely cause no conflict, allowing the initial configuration to survive. If there were other nodes that attempted to pub/sub on this topic simultaneously with the local node, they will use the same initial subject-ID configuration since they use the same hash and topic name, ensuring that every participant starts out with identical subject-ID mapping even before they are able to communicate. With every passing second and with every received transfer, the topic will age, and with age, it becomes more likely to win arbitration against possible contenders for the same subject-ID.

If the initially proposed subject-ID happened to collide with another topic, the first published gossip message will allow the nodes that know the conflicting topic to notice the collision. When that happens, the collision arbitration will be performed to decide who can keep using this subject-ID and who has to move; naturally, the topic with the smaller age will need to move. This is decided locally by each node that observes the conflict, and since they all share the same metadata for all topics they are interested in, they all will come to the same conclusion independently. When a topic is very young, however, a brief divergence is possible, causing a few ping-pong migrations until the age is settled. Regardless of the outcome, each node observing the conflict will immediately publish the new state, allowing the network to cross-check and synchronize the new state.

If the initially proposed subject-ID happened to differ from the values used by other nodes on the same topic (which would happen if there was a collision with another topic), then each node that is also on this topic will notice the divergence upon receiving this gossip message. In this case, a divergence arbitration will be performed to decide who is right and who has to update their records to ensure that all participants share the same subject-ID. As before, the older topic wins; given the same age (actually log-age, since we compare logarithms instead of raw ages), the topic with a greater eviction count wins because a greater eviction count implies that the topic saw more collisions and thus configurations with any lesser eviction count are non-viable (notice that this only holds if the age is equal; otherwise, a configuration with a greater age but smaller eviction count could still be viable because the greater age could allow the topic to evict the contender from the same subject-ID). The local configuration is NOT altered if (log(local_age) > log(remote_age)) || ((log(local_age) == log(remote_age)) && (local_evictions > remote_evictions); in this case the local metadata is gossiped ASAP to synchronize age and thus help other nodes on this topic win arbitration against contenders. This is the case when the eviction count may go backward, unlike the age counter, which only increments.