Fixing remaining deficiences in UAVCANv1 infrastructure

Development & Maintenance
#1

This thread is for:

  • Listing and resolving the remaining deficiencies in the infrastructure related to UAVCANv1. this includes but is not limited to documentation, application layer libraries, and most importantly, tooling.
  • Discussing the required steps for migration of standard platforms (especially Ardupilot and PX4) from v0 to v1.
  • Discussing potential additions to the application layer required for better adoption of v1 (example: automatic port ID allocation)

This thread is not for:

  • High level debates about the validity of v1 or differences from v0
  • Nonconstructive criticism about v1, the now defunct DS-015, or related work.
  • Discussion about the UDRAL specification or its implementation.
1 Like
#2

The things I can think of off the bat (roughly ordered by decreasing importance):

  • A fully functional Yukon that can be used for debugging and diagnosing v1 networks
  • A re-publication of v0 tooling (pyuavcan + UAVCAN GUI) for use in the transition to v1
  • Easy to understand, non-technical documentation about the differences between v0 and v1 and why a user should care
  • Demonstration of mixed-utilization v0 and v1 on a bus
  • Preferably, an end to end example of a project that utilizes the benefits of v1
  • libuavcan implementation of v1
#3

After speaking with some of the Ardupilot developers, it is clear that a demonstration of mixed-utilization of v0 and v1 is very high priority, as they are currently of the attitude that v1 is a nice-to-have that is blocked by backwards compatibility. To quote Peter Barker:

While we’d love to be able to support v1, co-existence with v0 is a pre-requisite, not something to believe-can-be-done

Additionally, James Pattison has requested that we consider forking the v0 artifacts into separate projects instead of old branches for easier access. @pavel.kirienko what do you think of this?

#4

I recognize that CLI tools are not as suitable to this end as a proper GUI tool, but still I want people reading this to know that Yakut is already there & usable on GNU/Linux and Windows (v1 only though):

The point of this is to permit concurrent installation of both v0 and v1 tooling into the same Python environment. There are two steps necessary:

  1. Republish PyUAVCAN v0 as pyuavcan_v0: https://github.com/UAVCAN/pyuavcan/issues/166
  2. Update the UAVCAN GUI Tool to use pyuavcan_v0 instead of the old package named uavcan.

This is not expected to be very useful as v1 can co-exist with v0 by design; making a demo is not expected to add new information. I am not against the demo though, just pointing out that there are other issues out there that actually need working on. Peter might be a little underinformed about v1, which I would be happy to help him with if necessary.

I don’t see why v0 implementations are considered to be hard to access, can you please elaborate?

#5

Here is one extra item that I believe should be on this list:

  • Formation of a new SIG dedicated to the physical layer connectivity standard.

This is expressly out of the scope of UDRAL because it is focused on the application layer. It is also out of the scope of v1 because reasons. I imagine that it should be mostly a no-brainer because we can just move the existing v0 hardware specifications over to the new standard as-is, amended with CAN FD support.

Should I go ahead and set up the SIG?

I think I should invite @PetervdPerk into this thread.

#6
  • Formation of a new SIG dedicated to the physical layer connectivity standard.

Indeed I agree this should be seperate from UDRAL, if there’s going to be a SIG for the physical connectivity standard I’ll gladly join to provide feedback.

  • Demonstration of mixed-utilization v0 and v1 on a bus

I’m a bit mixed about this, yes this works and you can easily demonstrate this. However it will impact the determinism of both v0 and v1 implementations due to the nature how CAN ID arbitration works.

#7

Should I go ahead and set up the SIG?

Go ahead. I expect this to be essentially a copy of v0 and/or DS-015’s hardware section, and I have no complaints or comments here.

This is not expected to be very useful as v1 can co-exist with v0 by design; making a demo is not expected to add new information. I am not against the demo though, just pointing out that there are other issues out there that actually need working on. Peter might be a little underinformed about v1, which I would be happy to help him with if necessary.

@pavel.kirienko I understand, but I don’t think you realized the value placed by the Ardupilot team on this effort. To quote @tridge (on the AP discord):

full co-existance with v0 is absolutely essential. That means the libraries like pyuavcan need to co-exist so we can have tools that analyse mixed networks, that means that nodes running both v0 and v1 at the same time need to work, it means that flight controllers need to be able to accept a mix of v0 and v1 sensors on the same bus

Furthermore, he pointed out that he is still unhappy with the state of type safety in UDRAL. I have invited him to log on to the forum and give input.

#8

v0 is the active, in use protocol. Forcing devs to unnecessarily bounce between branches just introduces increased likelihood of errors/git-spaghetti and is slowing things down. It would make life easier for everyone to simply treat v0 and v1 as separate entities.

1 Like
#9

One common strategy of release management is to branch out major versions into dedicated branches and maintain them there. This approach is not new so I presume that a regular developer is not to be surprised by this. Say, one day (soon, I presume) libcanard v2 will come around (it is going to be UAVCAN v1 but with API changes), whereupon we will branch out the current master into v1 and continue maintaining it there. Using separate repositories to substitute for release branches does not appear to be a sensible maintenance strategy.

#10

@pavel.kirienko please don’t try to be clever or patronising. You know exactly what I mean. We both know you’re intentionally trying to make life difficult for v0.9 in the misguided hope that in doing so it might accelerate adoption of v1. It’s having the opposite effect and discouraging UAVCAN adoption altogether. Start listening.

1 Like
#11

I’m sorry if my response reads as patronizing, this is not intentional. I don’t want this conversation to digress so let me restate again that I don’t understand why the current release management strategy is considered to impede the usage of v0. Can you explain this, please?

#12

Simply put, your release and communication strategy for v1 is premature. Until implementation of v1 is resolved (assuming it can be), v0.9 needs to be a peer, as a minimum. Frankly until implementation is resolved, v1 should be in a branch so that implementers understand that it cannot yet be used for product development.

1 Like
#13

In retrospect, the transition could have been handled better, indeed. Changing our migration strategy in the middle of the process would only serve to increase the turmoil so we are not going to do that.

@coder_kalyan I moved this topic into the Development & maintenance category to keep things organized.

1 Like
#14

@coder_kalyan I moved this topic into the Development & maintenance category to keep things organized.

Sounds good.

In retrospect, the transition could have been handled better, indeed.

@pavel.kirienko The transition is not over. As we discussed on the dev call, due to the wide usage of v0 in the industry, we are obligated not to abandon it until the transition has been made. This can still be improved.

Frankly until implementation is resolved, v1 should be in a branch so that implementers understand that it cannot yet be used for product development.

I think this is unnecessary, but I agree with James’ suggestion that we treat certain v0 artifacts as separate projects. Tooling such as pyuavcan should be published under PyPI as two separate projects that can be installed simultaneously. libcanard should probably be split off, altho its a smaller and therefore easier to handle project. If you are uncomfortable with publishing “out of date” projects on the UAVCAN repository, I can publish them on mine for now (not sure if this is ideal though).

it is going to be UAVCAN v1 but with API changes

This is an entirely different scenario. libcanardv2 would replace libcanardv1, they would never be in use at the same time. Our current situation is different because libcanardv1 is likely to be used alongside canardv0 (uavcanv0 implementation) for purposes of migration and backwards compatibility that Ardupilot wants to implement.

#15

@pavel.kirienko I apologize if this sounds crude but it needs to be said. Quite frankly, you must make the UDRAL development and related work public. I understand your hesitation due to the degradation of the previous DS-015 discussion, but at the very least, make it read-only public. The reason is twofold:

  1. As @mrpollo said, transparency builds trust. This is an open source project, and people like things developed in the open, rather than presented as open after all the important decisions have been made and the comment period is over.
  2. You said yourself that you are reluctant to dictate how you want UDRAL to be laid out, that you’d rather allow others to guide the discussion. You are in fact giving the opposite impression by making the topic private - that you want to limit outside input on the specification, and that you may be still making all the big decisions.

Please strongly consider what I have to say. Even if you disagree, it may be beneficial from a “public relations” standpoint to make things public.

#16

No, it wouldn’t. It would provide clarity.

#17

Will do. While the consideration is in progress, I would like to introduce clarifications:

This will be worked on (see the ticket I linked earlier in this thread), but it does not require changes to the existing development flow.

That’s not quite how major versions work though, assuming we are talking about semantic versioning. Putting aside the specifics of UAVCAN, any project that releases a new major version is expected to keep the previous one (or several) maintained for a while in order to facilitate the gradual transition of its user base to the new version. The v0 implementation is like an LTS release, not inherently different from any other release aside from its life cycle.

I don’t think I said that.

What I said was that I would rather allow others to guide the technical design as long as it is in line with the fundamentals of UAVCAN v1. It was my intention from the start to make the discussion private. We chatted briefly about this with Andrew, too, and I recall him raise objections, which he later restated on this forum. I chose to ignore the objections because I value the quality of our discourse on this forum very much, and I am willing to go to great lengths to keep the bar high, even if it requires me to enforce unpopular measures. I am confident that in the long term, the ecosystem at large will benefit.

#18

What I said was that I would rather allow others to guide the technical design as long as it is in line with the fundamentals of UAVCAN v1. It was my intention from the start to make the discussion private. We chatted briefly about this with Andrew, too, and I recall him raise objections, which he later restated on this forum. I chose to ignore the objections because I value the quality of our discourse on this forum very much, and I am willing to go to great lengths to keep the bar high, even if it requires me to enforce unpopular measures. I am confident that in the long term, the ecosystem at large will benefit.

Ok, that is your decision. Sorry if I misunderstood.

That’s not quite how major versions work though, assuming we are talking about semantic versioning. Putting aside the specifics of UAVCAN, any project that releases a new major version is expected to keep the previous one (or several) maintained for a while in order to facilitate the gradual transition of its user base to the new version. The v0 implementation is like an LTS release, not inherently different from any other release aside from its life cycle.

Not exactly; what I meant to say is that v2/v1 would be an implementation detail, while v0 and v1 are different protocols. But either way, I do think we shouldn’t try to brush v0 under the rug right now.

#19

One more thing that needs to be implemented is a v1 compatible embedded bootloader. Kocherga AFAIK is still WIP for v1 - it would be nice to port this to both UAVCAN/CAN and UAVCAN/Serial v1 (serial pending implementation of libserard).

Personally I’d prefer this to be implemented in raw C for portability, but I guess C++17 works.

2 Likes
#20

PyUAVCAN V0 has been successfully migrated to the new package name pyuavcan_v0: https://pypi.org/project/pyuavcan-v0/. This means that it can be installed alongside pyuavcan (v1) without conflicting with the uavcan DSDL namespace in V1.

The master branch of the V0 gui_tool also reflects this new package name.

Both have been run through CI and unit tests. I also tried installing both, and the GUI tool seems to run. However they haven’t been extensively tested, so it’s possible that something may show up later. If anything doesn’t work, please let me know or file a ticket.

4 Likes