Data type regulation policy and membership fees

Development & Maintenance
9

Sorry, I still don’t understand why this would be a problem. Unless I am missing something, there are no undesirable legal implications. Are you concerned about marketing/image issues? If this is the case, perhaps your mental model of a data type definition requires adjustment.

The purpose of a regulated data type definition is to provide an abstract common-ground interface for a typical function available on the network. The name of the namespace where the data type is located does not affect its generality. You can think about it as a software API that provides access to well-defined functionality in a generic and abstract way (consider the list of common Java APIs provided by many different vendors, or the contributed Python packages included in its standard library). Vendors are expected to make use of regulated namespaces just like they make use of third-party libraries, tools, operating systems, and so on.

10

My? :slight_smile: I conducted a survey among staff and students. No one understands why you need to use types of other firms. The specification is just what is needed to create more abstract concepts. I repeat. Repetition of types with this approach is inevitable. in Java, Python. It will be the same here. If you limit the number of developers, you can immediately close this project. Need an abstraction.

1 Like
11

Another interesting point. There is a metric system SI. What to do with England and America? Arrange a race which company will make new types for itself? :slight_smile:

1 Like
12

Vendors will be able to create and distribute their own data types without limitations, just like it was in UAVCAN v0 (there are certain rules governing the usage of fixed port-ID though, but they do not affect types themselves). I think this point might have been lost in the above discussion.

This topic is focused on regulated data types, not just any data types. Think about them as standard USB classes – it is really a good analogy. Any vendor of USB equipment can create their own custom USB interface, but they can’t affect the definition of standard classes. To do the latter, one has to bring their proposals into the committee that maintains the USB specifications. Same thing is currently proposed for UAVCAN.

As a vendor, you can define an distribute your own data types as you see fit (mind the fixed port-ID assignment rules though), but if you want them included in the regulated set, you have to synchronize with us, the maintainers.

SI is an international system, so no special cases are needed.

13

I think there is truth in both views here in relation to using another company types.

While many companies will have no problem using a type within the namespace of very well known companies, I don’t think it is realistic to expect they will use types from the namespace of unknown or direct competitors.

Why do we need a direct relation to a name? Couldn’t the type ID be related just to the contents? Then namespaces will be just conventions to group associated abstract types. Even if company B uses a type originally created by company A, it will not matter because it can call it whatever it wants, and still be compatible with any other type with the same contents.

Is there any special reason this can’t or shouldn’t be done?

I see regulated types as just a way of specifying well known types (e.g. a timestamp), associations between types and protocols (e.g. types used in a file transfer protocol) and define protocol-specific semantics (e.g. this timestamp for this version of the protocol must have some minimum resolution to be valid).

14

Perhaps the use of private data types although creates repetition, it could aim to create a sense of customization in the specific application that a particular company is building.

15

The reason we went with vendor-named namespaces is because this naming policy seemed clear and self-explanatory. I personally find the concerns expressed here so far insubstantial, but we are certainly open to new ideas. Could you (or anybody who’s reading this), perhaps, formulate a different naming policy proposal? You may find the following discussion relevant:

16

I have no problems with whatever namespace policy is decided, including having private company namespaces.
My point was that if we decoupled the actual structure type from the name (by, e.g., hashing the structure without any names or comments) then there would be no disadvantage in having private namespaces which were just a copy of other namespaces.
Names in the namespace would end as just a mapping to an abstract structure, because even if they were defined by different DSDL (with different field names, but same structure), the serialiser/deserialiser would be the same whatever the namespace, so we would end up saving on generated code and be able to have different companies having their own names for structures which would be compatible between them.
As a further advantage, after a used structure proves to be well established and used by everyone, it can move to a regulated uavcan namespace and everything would still work because it is only a “metadata” change.

17

Sure. What you are describing has been available in v1.0 since day one. You can find the details in the Specification:

18

Sorry, I did read an early draft of the 1.0 specification but seems I forgot most of it by now
We are using v0 on some of our internal projects. Have been waiting for some free time to start migrating things to 1.0 but lots of other more important things to do…

1 Like
19

At the dev call yesterday we’ve been discussing the problem of data type regulation again with @scottdixon, @TSC21, and @dagar.

The conversation was revived due to the ongoing work on the standard UAV profile spearheaded by the PX4 team. Their objective is to define a reusable vendor-agnostic set of well-defined standard data types and protocols on top of UAVCAN v1.0, as we intended:

image
image1458×742 57.2 KB

Our challenge here is to ensure that the UAV profile can be integrated sensibly into the larger UAVCAN (U for uncomplicated) ecosystem with the following objectives met:

  • UAVCAN-enabled hardware and software from different vendors are interoperable. This implies that the standard types are discoverable and vendors are not unnecessarily incentivized to rely on custom (unregulated) definitions.

  • The functionality addressed by regulated data type definitions is not unnecessarily replicated. It is acceptable and natural for a newer definition to obsolete its previous revision, but it is undesirable for the regulated set to contain multiple definitions addressing the same use case in incompatible ways.

  • Where appropriate and sensible, data type definitions are reusable across application domains. For example, manned electric aircraft and heavy UAVs typically leverage similar propulsion systems and power management hardware; likewise, the sensor suites of a car and an autonomous UAV may overlap. Cross-domain reusability would be beneficial for the larger UAVCAN ecosystem.

In the search for a sustainable data type regulation policy, we came up with the following options to organize the public regulated data type repository.

Segregation by owner

There will be one namespace under regulated per vendor/project. This proposal is currently in effect and it’s been discussed above in this thread (although, naturally, it is not final, otherwise we wouldn’t be having this discussion here). I consider it the most viable among the alternatives.

Under this model, the data types that are being worked on by the PX4 team would be published under the namespace regulated.px4 or similar. Having PX4 in the name does not imply that the types are specific to that project only; rather, it implies that they are the original author and initiator of the design.

Shall it be necessary to borrow definitions into an adjacent namespace despite everything, it should be possible to do by symlinking types or entire namespaces across directories. Remember that neither the name nor location of a type affects its wire compatibility.

Obviously, licensing issues have been considered: all regulated data types are released under the MIT license so reuse is possible with virtually no restrictions.

I would like to solicit opinions from the ArduPilot camp: @tridge @jani.hirvinen.

Segregation by application domain

There will be one namespace under regulated per application domain. For example, the types developed by PX4 would go under regulated.uav.

This arrangement creates obvious cross-domain reuse issues. To some extent, they may be addressed by symlinking, but the type discoverability issues persist.

Segregation by generation or some other abstract property

This is idea is new and rough. At the call, it’s been proposed to shard the regulated namespace by a more abstract property. Assuming that we failed to come up with a well-reasoned solution now, a namespace named after no particular entity could be chosen to contain all definitions we find relevant at the moment; for example, regulated.a. If this approach survived the initial trials, its successors could continue the trend with regulated.b, regulated.c, etc.

This approach risks making the data type set unmanageable in the long run.

No segregation

A flat structure like in ROS. For example, a particular BMS status message would go under regulated.sensor.bms.BatteryPackStatus.1.0 (currently, it’s under regulated.zubax.sensor.bms.BatteryPackStatus.1.0).

This is unlikely to work because we lack the required expertise.

20

There is one more scheme to consider (this is offered only to enhance the argument not as a proposal)

Segregation by device class

Similar to USB-IF classes, we could segregate interfaces by the hardware design goals. This is similar to the no segregation proposal but is flatter and requires each class to be developed by expert working groups with type evolution occurring at the device-class level instead of within an ecosystem as a whole. This means we would not accept a blanket specification for “UAVs” into the regulated set but would instead accept standardization for specific device classes. A drone system could then standardize on a set of regulated types but such a grouping of supported device classes would be a higher-level standard on top of the lower-level regulated types. Such leveling of class and group would naturally allow cross-domain sharing of types.

For example regulated.class.bms_light would be for a class of battery management devices designed for, relatively, low voltage (i.e. < 60v) and lightweight vehicles. This allows for a class of regulated.class.bms_heavy to exist for battery management devices designed for high voltage and/or heavier-weight vehicles like cars and trucks. In this scheme we are careful to describe the device class abstractly before assigning a class under which the working group can standardize specific types. The design goals for the hardware class are described by the type to ensure the proper trade-offs are made when designing types. Now add an example like regulated.class.esc_rotor and regulated.class.esc_wheel (to allow different interfaces for esc’s where microsecond responsiveness to control input is critical versus esc’s where power efficiency is paramount) and a UAV group would start with “The UAV profile supports the following UAVCAN device classes: bms_light, esc_rotor. …”. This allows manufacturers to choose appropriate interfaces based on the hardware they are developing and system integrators to limit the device classes their primary controllers have to support. As with USB, we would expect more powerful and complex systems to support greater numbers of UAVCAN device classes and smaller, more specialized systems to support fewer.

Base Classes

One other tool to consider is to utilize UAVCAN 1.0’s support for message extensions by defining generic base classes that the concrete types must extend. For example, a regulated.generic.bms type would provide an ability to interact with regulated.class.bms.light (now using dot-notation to acknowledge the existence of the bms baseclass) devices without utilizing any of the specialized “light” types. We would be aggressive about keeping the baseclass completely generic with some, possibly, containing no fields at all (i.e. since these would be considered abstract base classes). In such an eco-system it might even be a good idea to extend node.GetInfo to include a concrete device class type and base class type which we would assign (again, like USB 8-bit device classes and sub-classes). We could even go deeper down the USB rabbit-hole and add device protocol to allow versioning of the device classes (e.g. bms==0a, bms.light==01, protocol==01, meaning "this device is a lightweight bms supporting the first revision of the bms message set (yeah, that’s not exactly how USB protocols work but this is how I’d use it for UAVCAN)).

21

I think the following would be a nearly equivalent restatement of your proposal: you suggest to introduce a new namespace under regulated dedicated to data types that are considered sufficiently stable and generic to be recommended by UAVCAN maintainers for cross-domain use. When you say that such types have to be developed by expert working groups, that seemingly implies that the types have to survive early testing somewhere outside of the regulated set. Following the example of vendor-specific CSS extensions, we could combine segregation by device class with segregation by the vendor. I think this model would be easy to explain to everyone who has at least cursory familiarity with CSS.

Suppose we retain segregation by owner as described in my previous post and introduce an additional namespace for such expert-defined namespaces, which we could name regulated.endorsed (a shorter name is needed perhaps because we’re limited to 50 characters per full type name but I couldn’t come up with a better word yet). Once a particular namespace is deemed adequate for inclusion into the endorsed set, it is either moved there from an owner-specific namespace such as regulated.px4, or just symlinked directly. The idea of introducing base types first also seems very sensible and can be implemented here as well, such that the endorsed set supplies only the most vital functions, and owner-specific types can extend it as necessary. The endorsed set can also subtype its own definitions to support different degrees of generality, such as in your BMS light/heavy example.

Should we keep going in this direction?

I’m not sure I agree with the GetInfo idea because a single device may implement multiple profiles. We could allow devices to report several profiles at once but it seems like another major can of worms that is perhaps best kept sealed until some other day.

Edit: the scope of the endorsed set should extend beyond just standard function nodes and also include common concepts that are frequently referenced in various robotic and vehicular systems, such as various mechanical conventions (e.g., twist, pose), electrical conventions (e.g., DC power), optical (e.g., camera image, point cloud), and so on, that are not necessarily grounded to a particular piece of equipment.

Edit edit: in line with the above idea we may consider moving uavcan.si into regulated.endorsed.si.

22

I don’t like “endorsed” because it will be strange to have deprecated types under endorsed in the future but we don’t want to change the namespace when a type goes into deprecation. Similarly, when we have multiple versions which version is “endorsed”.

As for the need to “bake” APIs, I agree this is a great plan but to get the standard started we should allow a v1 set of types that are proposed by experts but which are allowed before field-testing. As the standard sees greater adoption and deployment we can improve this process for future revisions of the types.

Regardless, it seems from your response that you do not like my proposal as presented?

23

I think what you described makes sense, I merely attempted to generalize it further. I think there might be cases where segregation by device class only might lead to undesirable over-specification.

Consider a battery management system. The key reported property here is the amount of energy available in the battery which is used for the endurance estimation; the kind of the energy source is in most cases irrelevant for the consumer. Similar messages can be applied to other on-board energy sources beside batteries, from fuel cells and turbogenerators to nuclear reactors. In this case, one might want to abstract the core physical process (power delivery) away from the particular means facilitating it (battery, turbogenerator, fuel cells, etc).

Under that example, the namespace that segregates definitions by the type of the physical process would define a message containing a few fields like the timestamp, current, voltage, available energy, and the total energy. A message specific to a battery-electric vehicle (like a typical light UAV) would inherit from that and add BMS-specific fields like cell voltage and so on. Another message specific to a turbogenerator would add RPM and fuel levels, etc.

Thus the name we chose for that sub-namespace should be generic enough to allow for different segregation criteria. If endorsed doesn’t fit, we could consider regulated.uavcan unless there are other proposals.

Am I making sense here or did the conversation drift too far from the original goals? I think the key problem with the segregation-by-vendor approach is that some adopters might be reluctant to rely on definitions authored by some other parties for various non-technical reasons. I suspect that a particular open-source autopilot team might be particularly hard to persuade to adopt data types residing under the namespace named after another open-source autopilot project, if you catch my drift. By endorsing particular well-established interfaces we could act as a mediator here. This is not even a technical problem we’re solving here, if you look at it this way.

24

Any scheme will be subject to the infallibility of humans but the class-based scheme, as proven by USB itself, allows for mistakes to be corrected by defining new classes and deprecating old ones.

For your BMS example, the type could have a concrete baseclass that is generic. Certain systems would only require/consume generic BMS subjects but more specialized systems could extend this type to require more advanced subjects be present.

I’m fine with regulated.uavcan. I think we discussed this in the past but thought that having uavcan.foo and regulated.uavcan.foo might be confusing however I think this is tolerable.

25

Are you saying that there may be classes that describe more abstract entities than a particular type of device? For example, may there be a class for “power delivery”, which could be further specified as “battery power delivery” or “fuel cell power delivery”?

Yes, that’s exactly what I meant.

Seeing as neither of us are quite happy with the name, we should keep this question open for now, having regulated.uavcan as the fallback option.

26

To recap today’s call: @lorenz.meier believes that the policy of segregation by owner may cause issues because the adopters will probably fail to recognize the interoperability benefits offered by UAVCAN. He is in favor of segregation by application domain, where the problem of limited expertise is to be addressed by well-organized working groups.

The two approaches do not necessarily have to be mutually exclusive. A (sub-)namespace proposed by a vendor could be symlinked directly into a domain-specific namespace as necessary.

FYI @scottdixon.

1 Like
27

This a somewhat half-baked idea, but maybe I’ll throw it here:

How about allowing type definitions to be “content addressable” by hash, similar to what unisonweb.org is doing for code ?
This way vendor A would should have no qualms in copying the type of vendor B, calling it a different name but still be interoperable.
(this this doesn’t address the issue with structurally identical types with different semantics, but maybe this can be resolved by a very minimalist list of “compatible types” rather than screaming all over the codebase “I’m importing competitor’s code”)

28

Unison is an interesting thing but I can’t immediately see how its ideas can be applied here. For now, we decided (at yesterday’s dev call) to adopt the segregation by application domain where the issue of competence is addressed by the dedicated UAVCAN Drone SIG, which may be followed up by other SIGs in the future. We are going to see if it works out or not.