Simpler and better DSDL versioning

Development & Maintenance
1

Simpler and better DSDL versioning

I think this suggestion also is the best of both when it comes to code compatibility vs fixing field names and layout

Summary

Instead of versioning DSDL types individually we version the namespace as a whole. Rules to make this work better than the alternative will be set in place. This allow things as protocols that use several data types to be well defined.

This is also a more complete proposal including practical solutions for storing/releasing/managing standard and vendor specific DSDL.

Motivation

There are a few motivating factors for this proposal. The most important is that when creating protocols that use multiple types there is no way to express how the types rely upon each other. With this proposal types in the same major version namespace must work together, in addition they can be sustituted with any version they are @compatible with.

There was a big discussion what changes should be allowed “between” minor version bumps. This suggestion allows a more natural workflow, changing the type as needed and releasing versions when happy.

This suggestion is also a “best of both worlds” in regards to code compatibility. Allowing the breakage of code compatbility while remaining bit/semantic compatibility without introducing minor versions. The only inconvinience is that code breaking changes must be scheduled, which is a blessing in disguise as it gives users more predictibility of when code maintenance is required.

Description

The organization of DSDL

This section describes operational concerns around DSDL. None of this should be put in the specification. This section is included to demonstrate that this solution work well in practice.

For the time being, we’re using Github to distribute code, dsdl and documentation. This proposal will work with Github but also not be problematic to move to other solutions, git based or not. The only requirement is that it’s possible to create separate repositories and run automated checks on them.

We create a new GitHub organization to namespace DSDL where every repo, except those starting with an underscore, will be a root namespace. Initially, only the Uavcan organization controlled “uavcan” namespace will exist but other organizations can apply for their root namespace as well. The relevant branches of these repos must be protected so only changes will be accepted through pull requests.

One of the repositories in the DSDL organization, that is one of the previously mentioned exceptions to root namespace, will be called “_applications”. Anyone can fork this repo and add their application (using a template) for a namespace or vendor Port ID. Obtaining a namespace should be relatively easy, but require some documentation while ID’s would already require a namespace and a bit more justification. An application can be discussed in the PR and the PR can be merged when the application is accepted. That way there will be a public registry of all applications for namespaces and IDs available.

Some CI system (like Travis CI or Gitlab CI) will be used to verify certain properties of the PRs. This way we have the possibility of giving certain key people rights to merging PRs to a namespace they own (as long as tests passes). However, their rights must be limited, as force pushing or merging when tests failed would be dangerous.

Furthermore, all DSDL repos should (at least) be dual licensed under Apache2 and MIT to maximize compatibility and community acceptance.

Versioning of namespaces

This section should not be enforced through the standard, only describes good practice. It should be mentioned explicitly as a non-normative part.

Every namespace is recommended to be versioned with a <major>.<minor> version number. A convention will be created where it’s recommended to keep a definition in the root namespace containing constants defining the version number and other convenient things. If versioning of the namespaces is used, it’s good practice to keep “version tags” in the repositories to allow easier fetching of a concrete version.

The official uavcan namespace will follow the previously mentioned recommendations.

Stability directives

Definitions do not have separate version numbers. Since things as early experimentation and tracking of long-term stability are important the following (meta) directives will be used.

  • @unstable - “unstable”
  • @compatible (<major> | 0.<minor>) - “compatible with versions (since or of the same tag)”
  • @deprecated - “deprecated”

@unstable

The @unstable exist for early experimentation, and a definition tagged with this directive do not adhere to any stabilization guarantee and may be changed in arbitrary ways only updating the minor version of the definition. A DSDL compiler should either not compile such definition by default or map it to language mechanism for unstable features.

@compatible

Since breaking fielded systems is something that can be damaging for the ecosystem. We introduce a @compatible tag as a first class citizen of DSDL. If versioning is used on some level (namespace or datatype), @compatible (<major> | 0.<minor>) can be used to express the oldest version in this system the definitions is compatible with.

Two definitions with the exact same namespace, fully qualified path and @compatible “tag” must be bit and semantic (not necessarily code) compatible.

This “little trick” is what makes this system, even though simpler and more complete, as powerful as “separate versioning of datatypes”. Instead of having the GetInfo service return a concrete version it will return a “Compatible since” version. If this “compatible since” version matches with your own version everything is good and jolly even though the major version of the namespace is different.

If the workings of a “micro-protocol” changes in a way that breaks semantically compatibility the “compatible since” version must be updated to reflect this. This can only be done during a major version bump, as minor versions must be semantically compatible.

@deprecated

A minor version bump is allowed to introduce the @deprecated directive. This schedule the type for removal at the next possible moment (if using namespace versioning, this will be next major version bump). This directive can peacefully coexist with the @compatible x.y and @unstable directives. But a type tagged with @unstable could also be removed without deprecation.

Allowed changes for a minor version bump

This section should not be enforced through the standard, it only describes good practice. It should be mentioned explicitly as a non-normative part.

To ensure that updates between minor versions can be done without fear. For a stable definition, every change that increments the minor version must be:

  • Bit compatible
  • Semantically compatible
  • Code compatible

Requiring code compatibility between minor versions is not a problem as the @compatible x.y directive can be used to express bit and semantically compatibility between major versions. This will allow the code that is produced also to be semantically versioned equal to the DSDL and be distributed conveniently.

Changes to field layout and such (that is semantically and bit compatible but not code compatible) must be scheduled during a major version bump, and mean that such change cannot be done “this week”. This system schedules major version bumps of the uavcan namespace regularly, to allow such small changes can be collected and applied synchronized so only a single code update is required for users. This seems to give excellent possibilities to improve things while giving users excellent stability guarantees.

In addition, removing the @unstable tag or adding the @deprecated tag is also allowed for a minor version bump. Changing the @compatible tag is not allowed as this is a breach of semantic compatibility.

Minor versions do not follow a strict release schedule and are bumped as the uavcan developers feel necessary.

Major version releases guidelines

This section should not be enforced through the standard, it only describes good practice. It should be mentioned in some sort of operation documentation, but not in the specification itself.

The organizations controlling a namespace are allowed to do major version releases as they please. This section describes good practices that they are recommended to follow. The Uavcan organization should do its best to follow this for the official namespace (uavcan namespace) but smaller divergences are tolerated when justified.

First of all, releasing a major version does not necessarily break fielded units or the ecosystem. Units are highly recommended to use the @compatible directive for compatibility checking of relevant types, and a bump in the major version should not break anything other than the concrete types that are broken. Even though a major version bump is technically free to break all the datatypes it wants to, this should not be done lightly. Stability of fielded units is extremely important, and breaking datatype compatibility must be avoided unless there exist excellent reasons for doing so.

The namespaces should follow a schedule for updating the major version rather than doing it unexpectedly. Even though fielded units or the ecosystem will most likely not break for a major version bump (except in cases where this is absolutely deemed necessary) the code is expected to break. Scheduling removal for non-functional changes, like deprecated types and doing changes that break code compatibility, is highly valuable as this code maintenance can be expected and done efficiently. These non-functional changes can be very nice, but is generally not something that needs to happen overnight, due to this adhering to a release schedule for major versions is something that seems to be acceptable.

The release schedule for major versions for the official uavcan namespace should be every three (3) years with the following exceptions. When version 1.0 is released, version 2.0 is expected to release in one (1) year after. And version 3.0 is expected to release two (2) years after version 2.0. The more frequent releases early on is due to the increased demand for small fixes early on. And the need for deprecating mistakes made early on more quickly.

Comparison to seperate versioning of datatypes

The arguments are denoted as following:

  • (+) in favor of namespace versioning
  • (-) in favor of datatype versioning
  • (=) neutral arguments or statements

Arguments for/against namespace versioning

I will add/remove arguments to this list as the proposal is being discussed.

  • (-) Changes requiring code compatibility breakage can not be done “any time of day”.
  • (=) Changes to fix names, field-layout etc can be done without breaking bit/semantic compatibility in both alternatives.
  • (+) Changes that require code maintenance is expected and scheduled.
  • (+) Less surprising, and more similar to what the rest of the world is doing.
  • (+) Allows easy and safe distribution of generated code as well as DSDL.
  • (+) Gives us deadlines to work towards, motivating continuous improvement.
  • (+) Semantic compatibility can be broken elegantly for micro-protocols.
  • (+) It’s clear when deprecated types can be removed.

Unresolved questions

  • Exactly what to call the DSDL github repo. I propose DSDL or Uavcan-DSDL.
  • What semantics should we use on the @compatible/@stable directive. Either “Stable since version x.y” or “Compatbile with version x.y and above”
  • Figure out the detail of handing out namespaces/IDs and how the templates should look.
  • Gitlab has good CD (continous deployment) integrated, this is in my experience much better than what travis has to offer. Should we consider using gitlab for development with a github mirror?
2

Even though i posted this now, I will take one more look at it tonight and perhaps adding some details.

I felt i had to post it now to give you guys a bit time to take a glance at it before the meeting.

3

I thought we agreed that breaking code-compatibility required a minor version bump whereas breaking semantic/bit compatibility required a major version bump?

4

This is pretty much equivalent to the old system as the @compatible directive keep track of bit and semantic compatibility from different major versions. When we want to break code compatibility without touching anything that goes over the wire we collect all desired changes and do it when a major version release is scheduled.

Allowing to break code compatibility when bumping minor versions would quite frankly not work well, as we’re actually versioning the whole namespace and we have to bump the minor version for the whole namespace for a minor change. Since we have no way of cherry-picking which versions we upgrade (like we had when all the types had a separate version) we will break peoples code too frequent for it to be acceptable.

What I mean by this is the “best of both worlds” is that by collecting changes and apply them at a major version we retain the opportunity of fixing everything that can be fixed without breaking anyone’s code for a minor version upgrade. It will require a bit more planning, but I’m sure this is only a positive thing.

5

There are several serious issues with this proposal, probably the most important one is that it flips the solution upside down, pushing the utterly important matter of wire compatibility (which is semantic compatibility and bit compatibility) to the background (via @compatible) while moving the far less important feature of code compatibility at the forefront. This is probably due to the implied underlying assumption that the application’s own code and its data types are to be versioned, released, and maintained in more or less the same way. This may be the case in deeply isolated ecosystems, but anywhere else this is not true. Data types represent carefully designed interfaces between different well-encapsulated computing systems; the interfaced computing systems are extremely unlikely to be maintained under synchronous release cycles, and as such, the release cycle of involved data types would be dictated by the least frequently updated computing system in the application (more on this: The case for code compatibility) which implies that compared to general-purpose embedded applications, data types are to be almost unchanging by design. If one needs to update their data types as frequently as the application itself, the data types are probably poorly designed (notice that the same could be said about software interfaces in general).

Another important issue is that this proposal makes simultaneous use of data types under different major versions unnecessarily difficult, and complicates static analysis. If one wanted to use several major versions of a data type in the same application, they would have to include several versions of the namespace. Since each namespace version is an isolated system, it would be difficult (although not impossible) to ensure that the definitions available in all of them do not conflict with each other. For example, a particularly troubling case is when a fixed ID of a data type defined in one version of the namespace is used for an incompatible definition of the same data type in a newer version. Admittedly, this is resolvable, but it just adds extra complexity for no good cause.

As we discussed at the last call, the specification cannot be dependent on any particular version control system, code hosting service, or any external infrastructure unless such infrastructure is exceptionally stable (i.e., as far as external infrastructures go, it is safe to rely on the English language or world wide web, because these things are not going away in the foreseeable future; whereas the long-term future of git, Github, or the Sami language is questionable). The specification also should not concern itself with trifling matters of access management, approval process, and all that related bureaucracy.

This is a significant departure from our current (i.e., old, the other one, not this one) draft proposal. For the benefit of other readers, I should clarify that it is mostly motivated by the problem of micro-protocols, which was first mentioned the old thread. The idea is that the current (old) proposal lacks adequate means of expressing semantic interdependency between different data types; currently, data types can express dependency only unidirectionally and only through composition: if A contains a field of type B (possibly indirectly), then A is dependent on B (possibly indirectly). What is missing is that A may not even mention B anywhere in its definition, while being semantically dependent on it; likewise, if A is a composition that contains B, it does not necessarily mean that B can’t be semantically dependent on A. There are several cases in the WIP standard data type set where the problem can be observed; yet I don’t think that it is severe enough to warrant immediate intervention. Several solutions can be discussed; the most obvious that come to mind are requiring that all interdependent types share the same major version (this implies that some data types may need to have their major version bumped while retaining semantic and bit compatibility), and adding special directives that fill the need for an explicit dependency declaration mechanism (like @depends foo.Bar, baz.*). I want to push back on that and release v1.0 without any such logic in place since it can be safely added retroactively. Doing that now not only delays the release (it’s no easy feature, and we will probably spend a few more months going back and forth unless we met in Tallinn/Trondheim/elsewhere for a day), it is also risky since we don’t yet have any experience working with the versioning system we designed.

Last but not least, the new proposal just doesn’t seem to map on the problem domain as well as the old one. Data types are more or less atomic interfaces (to some extent), even when they are used within micro-protocols. One should be prepared that any given non-isolated ecosystem may have dozens of nodes from different suppliers all using different data types or different versions of same types; this enforces a certain fragmented variety of the versioning problem that can’t be mapped well to the new proposal that assumes that namespaces or data type sets are inherently atomic.

Now, that said, I don’t really have anything against the general idea of root namespace versioning as long as it is done on top of the old per-type versioning solution and it is not mandated in any way in the specification (except probably in non-normative sections, but even that is probably unnecessary). This can be done trivially using standard principles of semver, and the protocol does not (and must not) care about it. I mentioned this approach in the old thread. It simply means that we can tag certain commits in the DSDL repo as “releases”.

Overall, I suggest we go back to the old arrangement.

6

I’ve updated the proposal with the following changes:

  • Explicitly say whats concerning the specification and what is operational guidelines.
  • Made it clear that git/Github is best bet for now, but not something we want to marry.
  • Made it clear that versioning of namespaces are strictly speaking optional, and there may exist ad-hoc DSDL definitions that are correct without having a version.

I feel this is the total opposite. The @compatible keyword is a first class DSDL citizen with ability to “override” the outside versioning system. The reason @compatible exists inside the definitions is simply because that’s the object it works on. We could have a compatible.json listing all of the compatibility concerns in the “foreground” but that’s not better at all.

Yes, using different major versions simultaneously should not be done. You must instead support different major versions separately. A system (a bus with many nodes) only use one version of DSDL at the same time, and the units should be configurable with this version. This probably means that we need a DSDL configuration service that we realistically cannot change (like the Heartbeat/NodeStatus message). Since major versions are released on a schedule there will be a limited number of versions that need to be supported even if support for the last tenfold of years is provided (if we extend the release cycle to 5 years, last 20 years can be supported with 4 versions, with 3 years cycles last 21 years is 7 versions).

This makes testing simpler as you have a total 2-10 versions to support instead of combinations of 1-4 versions for every data type you’re using. It also makes the configuration simpler as you only need to configure every node for the same DSDL major version.

I think what your suggestion is pretty close to what I’m suggesting (I suspect it was your comment that put me on this track). Perhaps the few edits I’ve made express this more clearly now. The main differences are that the data type version is replaced with a @compatible directive. I think this allows the things you’ve been wanting, without much penalty:

  • It allows changing comments and whitespaces wild willy (only updating the namespace minor version before “release”).
  • It does not require code compatible for keeping the definitions @compatible.
  • It allows disregarding everything except the core property of “over the wire” compatibility.
7

Just to see if I understand; this would be an optional, flat-file under the namespace folder similar to Python init.py (without the executable logic, of course)?

8

That was what I was thinking. This is what I would call “operational recommendations”, and the only reason I specified this at all was to show that there exists a convenient way to resolve the namespace version. Maybe there are better ways to do this, but it’s not really important for the proposal exactly how this is done, we can bikeshed it if the proposal is accepted.

9

We probably should discuss such major breaking ideas before building sophisticated proposals that depend upon them; that should save time and effort. Joint data type versioning like you described will not work because it obviously makes introduction of new data types impossible unless every single node on the bus is switched to the new namespace containing them. You should stop thinking about closed ecosystems, where that would probably be okay (at least until such ecosystems are small and simple) and consider a larger, more generic picture, where you may not have any control over the types used by nodes on your bus. Further, it strongly couples the release cycles and maintenance of possibly completely unrelated types (which are inter-node interfaces, it helps to think about them this way) that have to coexist under the same root namespace. Not everything under the same namespace is a micro-protocol. This latest proposal goes against some of the core ideas and just looks uncannily like the “identifier distribution” scheme used in CANaerospace and its derivatives; we do not want to go that road.

If separate per-type versioning is needed, then the version number must be contained in the file name; otherwise there may be several different definitions under the same file name, creating conflicting entries on the file system level. Another reason, which is at least as important as the first one, is that the version should be immediately visible when the developer is looking at the directory structure; it is the same reason why we have the fixed port ID (formerly known as data type ID) as part of the file name instead of using dedicated directives inside the definition. Plus it makes definitions somewhat simpler, although that is far less important.

There is one thing that is in my opinion worth discussing: whether we need the minor version on a per-type basis. I am inclined to think that it is useful and should be kept as in the old proposal because while it is guaranteed that different minor versions (under the same major version) are all semantically compatible, it does not mean that they are semantically equivalent (indeed, that would make minor versions virtually useless). Hence, the minor version number conveys useful information at least for diagnostic and analytical purposes.

10

It’s not impossible, (if it is, it’s at least not obvious to me?), you just up the minor version and introduce the new datatype. This is even possible (and trivial) after new major versions are created. Important additions can easily be backported into older major versions as long as they don’t break anything.

The only time when backporting is not a viable option is when an older method is deprecated and replaced with a newer method. But then all units on the bus must consistently use the older method, which I guess is better than some sort of “configuration hell” of which units uses which types.

We can bikeshed for how long, but It seems reasonable to consistently backport definitions for at least 3 releases in the uavcan namespace.

There is, of course, the problem that a node cannot use the new data type if the firmware was compiled before the data types was created. But this is more of an inherent problem with any sort of system and not special to this proposal.

I’m not at all worried about closed ecosystems in the DSDL setting. These systems are quite simple and most things will work here.

You should be a bit more worried about testability, maintainability, and usability of these types when applied to this big open ecosystem. We need good predictable tools and conventions to manage this complexity.

We need to show users that we’re responsible in regards to stability and testing if we want any sort of widespread adoption of DSDL. If we’re not, users will not dare to use DSDL and define their own types instead.

Not really, it just couples breakage of any sort (including code) with the release cycles. In most cases, these unrelated types can be maintained without breaking anything, making them decoupled.

The maintenance of unrelated types is only coupled when breakage is required. But that breakage only can happen rarely is an incredibly useful thing. It forces the maintainers to plan ahead, but not to the point that mistakes cannot be fixed. It still makes it possible to test a few different configurations (major versions) of types and know that these are the configurations are the ones that will be used.

Please elaborate?

I think a lot of your critique of this proposal has been based on misconceptions. I still think this proposal is the better alternative and I think we should dig a bit deeper.

Please restate critique I’ve failed to answer to.

Also keep in mind that even though your problem of multiple namespace versions “at the same time” needs a robust solution, most units would be able to look at their used data types @compatible tag and conclude they’re compatible with many namespace versions without needing any form for configuration.

Even those requiring configuration would probably only require two or three settings, spanning a long range of multiple versions.

11

The following core ideas should be accommodated:

  1. It must be possible to introduce a breaking change (in the sense of wire compatibility) in one part of a namespace (by releasing a new major version or similar) without affecting other types in the same namespace. Different types serve different purposes, regardless of the fact that they are in the same namespace; a message related to flight control cannot be sensibly versioned synchronously with a file transfer service. The @compatible directive does not seem to solve this in any way because the release of a new major version requires all participating nodes to be aware of the new version of the namespace because nodes “must support different major versions separately”. Further:

The @compatible keyword is a first class DSDL citizen with ability to “override” the outside versioning system.

This is a crutch. As you are descibing it, it is necessary to tie the solution to its problem domain because it doesn’t fit well on its own.

  1. Breakage affecting the whole namespace without a very strong reason (such as releasing a new version of the entire protocol), scheduled or not, is unacceptable. That would undermine the long-term stability of the protocol; this is not a JS framework.

  2. Concepts pertaining to code compatibility, code breakage, and code maintenance should not appear in the (normative parts of) specification, because they are too low-level and are not directly related to communication. I have introduced an alternative idea (that would lie out of the scope of the specification) where entire namespaces can be versioned purely for the sake of code compatibility and maintenance; if desired, we can go back to that.

I believe these were discussed at length in the past; specifically, the vices of global versioning are covered here: https://github.com/UAVCAN/dsdl/issues/35, and why code compatibility is irrelevant was discussed here: The case for code compatibility.

This proposal essentially re-raises the same questions, albeit posing them in a different way. Despite the new outlook, the core arguments remain unaffected and are still valid, I don’t think repeating the old discussions here makes much sense. It is clear that the new proposal contradicts the key ideas, and unless there is a sensible proposal to discard them without affecting the core design goals of the protocol, discussing this further does not make sense.

Observe that the old versioning proposal seems to fit the requirements well; unless the same can be shown about this one, we should go back to the old proposal.

12

I think the case is presented. If you really think this proposal doesn’t make sense, I will refrain from discussing it any further.

The core part of the uavcan protocol v0 is very solid, with the improvements we’ve made I’m certain v1 will be even better. I’m very confident that people are going to want to use it.

The DSDL definitions in v0 was flawed, even though the improvements in v1 makes (some) things better, I’m still not as confident in this part of the protocol as the core. I’m not saying it’s the wrong decision to move forward with the current design. But with the current imperfections I cannot honestly say I would want to use the “official” DSDL , and would probably end up defining my own custom types. Perhaps the problem is that I’m not a representative user of the protocol, and if that’s the case it’s totally fine.

I guess unless others have concrete proposals for improvement the constructive thing is to move forward with what we got and hope to gain more insight during testing.

13

Thank you, Kjetil. I urge others, especially Scott, to join the discussion as well, and share their insights. We will be able to discuss this somewhat at the tomorrow’s weekly call.

14

I feel like you are debating too many points at once and I’m having trouble following the arguments. Let me try to understand each part one at a time. Let’s start with the most visible aspect of these proposals:

Granularity of Versioning

Pavel’s proposal is to version all datatypes individually. So if we had a set of types like this:

uavcan/weather/Location.1.0.uavcan
+---------------------------------------------------+
| float32 longitude
| float32 latitude
| float32 elevation
+---------------------------------------------------+

uavcan/weather/65501.Observation.1.0.uavcan
+---------------------------------------------------+
| uavcan.weather.Location location
| uavcan.time.TimeSystem  time
| float32                 precipitation_cm_last_hour
+---------------------------------------------------+

uavcan/weather/65502.Forecast.1.0.uavcan
+---------------------------------------------------+
| uavcan.weather.Location location
| uavcan.time.TimeSystem  time
| float32                 precipitation_cm_next_hour
+---------------------------------------------------+

… and wanted to change the Forecast type you would only modify the Forecast type:

uavcan/weather/65502.Forecast.2.0.uavcan
+---------------------------------------------------+
| uavcan.weather.Location location
| uavcan.time.TimeSystem  time
| uint32                  duration_hours
| float32                 precipitation_cm
+---------------------------------------------------+

Kjetil’s proposal is to only version namespaces. With this proposal we’d have something more like this:

uavcan/__namespace__.uavcan
+---------------------------------------------------+
| @semver 1.0
+---------------------------------------------------+

uavcan/weather/Location.uavcan
+---------------------------------------------------+
| float32 longitude
| float32 latitude
| float32 elevation
+---------------------------------------------------+

uavcan/weather/65501.Observation.uavcan
+---------------------------------------------------+
| uavcan.weather.Location location
| uavcan.time.TimeSystem  time
| float32                 precipitation_cm_last_hour
+---------------------------------------------------+

uavcan/weather/65502.Forecast.uavcan
+---------------------------------------------------+
| uavcan.weather.Location location
| uavcan.time.TimeSystem  time
| float32                 precipitation_cm_next_hour
+---------------------------------------------------+

… to make a similar, breaking change to Forecast we’d end up with this:

uavcan/__namespace__.uavcan
+---------------------------------------------------+
| @semver 2.0
+---------------------------------------------------+

uavcan/weather/Location.uavcan
+---------------------------------------------------+
| @compatible 1.0
| float32 longitude
| float32 latitude
| float32 elevation
+---------------------------------------------------+

uavcan/weather/65501.Observation.uavcan
+---------------------------------------------------+
| @compatible 1.0
| uavcan.weather.Location location
| uavcan.time.TimeSystem  time
| float32                 precipitation_cm_last_hour
+---------------------------------------------------+

uavcan/weather/65502.Forecast.uavcan
+---------------------------------------------------+
| uavcan.weather.Location location
| uavcan.time.TimeSystem  time
| uint32                  duration_hours
| float32                 precipitation_cm
+---------------------------------------------------+

Am I representing this correctly?

15

I think the syntax may not be exactly what Kjetil intended, but the idea is right. Also it doesn’t have to be limited to the same nested namespace weather; there could be navigation alongside it, and it would be versioned synchronously with weather.

16

This is not what I was suggesting!

Every root namespace has a version. There are no hiarchy of versioning beyond that. Meaning uavcan would have a version and uavcan.x uavcan.y would always share the same version when used together.

That is the reason only a single switch is needed per root namespace in use (which for many simple systems would only be the uavcan root namespace). And the testing done on a root namespace version would be representative for what would happen in the field.

17

This is what I meant when I said that they are versioned synchronously. Meaning that there is the same version number(s) applied to all items under the root namespace.

18

Okay, I updated my example.

The result of this proposal is that after the first version change every datatype would receive a @compatible directive, correct?

19

I’m sorry. It’s just very important that they’re not only synchronous but also equal. Using synchronously updated but unequal versions would be unnecessarily messy.

Yes, that’s correct. It could technically also be introduced already at version 1, as they would be trivially compatible with version 1.

As an example:
If you’re using major version 5, and all the types you’re using is tagged @compatible 3 (more precisbly 3 or older) your software would be compatible with buses ruinning 3, 4 or 5 without needing configuring. Perhaps by using version 2 you could create a software that would be compatible with 2 and 1 as well, effectively needing 2 configurations to support all existing versions.

When new types are introduced in recent major versions they can more often than not be tagged @compatible 1 as they will most likely not conflict with anything in the older definitions. These types are then backported into the older versions to make sure conflicting versions cannot be introduced there at a later point.

The more complex the unit is the bigger chance is it to fragment compatibility into mulitple ranges. A unit using every DSDL type might need a seperate configuration for every major version. If we decide to set the release schedule to 5 years it would still support 20 year of DSDL with 4 settings. With the per type version system the number of configurations is not constrained by time, instead requiring multiple settings for every type in use.

Even though this maybe at first glance can look like like a fragmentation danger, the hope is also that when you only need to track one number that changes once every X years keeping up is less overwhelming and more likely to happen.

20

I think, in a way, what we are proposing here is ultimately an API version spanning each root namespace. This reminds me of Android and Google Play Services versioning as a similar problem domain: where the Android API version updates on a yearly schedule and the Google Play Services APIs update as these services are introduced and modified.