Proposal: application layer libraries

Development & Maintenance
#1

Libuavcan is quite a large framework which will take a lot of work to migrate to V1 - and so far, I don’t believe that anyone has offered to do the work.

To this regard, I have the following radical proposal:

Either to replace (or in addition to if we don’t want to let go of libuavcan), I propose that we create small, C-based application layer helper libraries for different use cases.

For example, there would be something on the lines of libucanapp for the standard application layer, libudral for UDRAL logic and PnP allocation, etc.

Benefits:

  • small
  • modular
  • provides the batteries-included application layer logic without a lot of added baggage
  • C implementation, no requirement on C++. C++ support could also be provided if people want it.
  • provides a consistent, cross-ecosystem reference implementation of application layer help for projects that want it - reducing the amount of testing/validation that needs to be conducted on e.g. autopilot implementations.

The only few downsides I see:

  • people who want to migrate from libuavcanv0 will find no 1:1 alternative
  • people who want a C++ API may be disappointed
  • Abstraction for the physical layer (e.g. socketcan) will have to be moved somewhere else.

Thoughts?

#2

I think @scottdixon did, alhough he did warn us that unless someone comes along to lend him a hand, it’s going to take some time.

Can confirm. Am disappointed already.

May I suggest that we refocus on libuavcan instead? I think @scottdixon should be able to indicate well-defined entry points for new contributors.

#3

@pavel.kirienko I understand. Perhaps there’s some merit in splitting them into multiple (C++) libraries though? Will discuss at the call.

#4

libuavcan v1 has a defined “media” layer and that’s as far as I’ve gotten. I’m still picking away at Nunavut C++ serdes which will be forthcoming.

Anyone that wants to dive into completing libuavcan would be most welcome and I’m willing to offer any support I can.

That said, there is also merit to abandoning C++ for the “batteries included” reference implementation. As much as C is relic it is more portable and avoids issues like those raised by DO-332. I’m not opposed to discussing this option.

1 Like
#5

I wanted to add a disclaimer: as much as this may be an unpopular opinion, I personally much much prefer C as a language to C++.

Personal opinions aside, C has the benefit of being ultra portable and being future-compatible with most other compiled libraries (C++, Rust, Zig, Python, etc). TL;DR: It is easier to write a C++ application using a C library than write a C application using a C++ library. C++ suffers somewhat in this regard. It is also not very difficult to add a thin C++ wrapper over a core C API.

@scottdixon what do you think?

#6

What we may lose by not having libuavcan is a vertically integrated solution for developing UAVCAN applications in C++, including the physical layer. Whether or not this is necessary, though, is up for discussion.

1 Like
#7

I don’t like making aesthetic decisions about software languages so I’ll avoid stating a preference. Your portability argument is apt and is the primary reason C persists and will continue to persist. That said, C falls down as complexity increases. It’s a great systems programming language but a poor application programming language.

C++ is a terrible, terrible language that is the best low-level application programming language widely considered viable for use in high-assurance software. Starting from C++11 and forward, the ability for developers to express constraints that are statically checked is a strong argument for its use. I don’t have personal experience with any other language that is able to do as much compile-time validation as modern C++ (although I’m told Rust is even better). There are, however, two critical problems with modern C++ as an application framework:

  1. The template syntax is comically hard for humans to understand. I think anyone with any experience with C++ would concede this is true. For an API, being syntactically abstruse is a particularly poor property.

  2. Templates complicate software verification and can inhibit certain certification options. Specifically, if you wanted to verify a given software library as a reusable component the use of templates makes this almost impossible. For example, if you wanted to obtain a certification for libuavcan such that it could be used without reverification in other certified software projects you would be providing a hash of the binary verified as part of your certification material. Templates, however, mean that the hash changes for each use of the API thus invalidating the verification material provided.

So there are tradeoffs for choosing either language (I could go on more about how C code is, ultimately, less optimized for a target platform but this gets into some very detailed arguments we don’t need to have). What’s more important is the functionality. If you are available to develop a full “batteries included” UAVCAN v1 implementation suitable for use on both bare-metal firmware and modern operating systems then I am eager to accept your contributions.

4 Likes
#8

It might be a good time to mention that I find debugging in libuavcan for v0 extremely difficult. I much prefer dealing with the simple API in libcanard. The number of levels of abstraction in libuavcan is just far too deep.

3 Likes
#9

I agree. What the infuriating obfuscation buys, however, is a lot of information the compiler can use to produce a carefully checked and optimized binary. Again, it’s not great for humans but machines love it. C code, on the other hand, will lead you along like a lullaby until you hit some undefined behaviour that takes the next 48 hours of your life but which a C++ version of the same code would have found in under 5 seconds at compile-time.

I should say, there is a middle-ground. By writing C++ but rejecting any template meta-programming or other templatized optimizations that are unreadable, even at the cost of less efficient runtime code, you can get the best of both worlds. Namely, c++ casts are an important improvement over C; type_traits and static asserts allow for stronger API contracts; references, std::unique_ptr, and std::optional all but eliminate the need for raw pointers; move semantics preserve memory bandwidth without constant hand-optimization by the developer; etc. Paired with a typical embedded profile (no exceptions, no threads, no threadsafe statics, no RTTI, no heap, etc) C++ is capable of being even more human friendly then C but it takes discipline and experience.

#10

libuavcan? optimized? I think you must be kidding :wink:
We had to completely restructure ArduPilot CAN support at one point to cope with the templates of libuavcan resulting in huge chunks of code being duplicated multiple times in the binary. Each time we added a new subsystem that supported UAVCAN (eg. AP_Rangefinder, AP_Baro, AP_GPS etc) we found that the firmware grew by a mysteriously large amount. We now structure libraries/AP_UAVCAN differently to avoid the issue.

I completely agree, as long as C++ is used well and you know about some of its nasty corners. ArduPilot is written in C++, and I’m very happy with how C++ has given us some really nice safety over C code. We’ve even adopted some of the ideas that I first learned about from libuavcan, for example the ArduPilot WITH_SEMAPHORE() macro was inspired by the equivalent in libuavcan (a clever constructor/destructor stack variable trick)

yep, I completely agree. I just find libuavcan to be a bit beyond my comfort level. It is one of those “here be dragons” subsystems that only a few people in the ArduPilot dev team are willing to dive into (namely Sid and myself). Both of us tend to grumble when we need to dive in there :slight_smile:

1 Like
#11

the other things I really miss from C:

  • a tool to get the maximum stack usage for a call tree, so you can sanely set -Wframe-larger-than= for embedded work where you don’t have the luxury of a MMU
  • a compiler that does link time optimisation well enough for production use
1 Like
#12

Yeah, it’s pretty aggressively optimized to use less memory at the cost of code size, in my experience. This has been a good trade for some of the MCUs I’ve been using recently since they tend to come with lots of flash but not nearly enough SRAM (When you start using CAN-FD this lack of ram becomes even more acute).

Just a shout-out to @pavel.kirienko for developing this library in the first place as we are writing down lessons learned. Without the experiment the improvements wouldn’t be so obvious.

1 Like
#13

@scottdixon @tridge Sorry to interrupt a very interesting conversation but I think we’re starting to lose sight of the original intent. My intent was not to start a discussion about the merits of C versus C++, which is faster or more optimized. I only had 2 specific points with regards to an embedded implementation of UAVCAN:

  • I am pointing out that C is more portable, and therefore has a chance to be applicable to more users/vendors/projects. I merely pointed out a thin C++ wrapper to satisfy those who don’t want a raw C API running loose in their C++ project.
  • Independent of C/C++, I think there is some merit to having a set of loosely connected, simple APIs as I suggested above, instead of a vertically integrated solution like pyuavcan/libuavcan. From what I can see, the use case for a C/C++ library seems to be the embedded scenario more than the application layer scenario, anyway.

If either (or both) of these points have any merit in your opinion, I would be interested in working on them (after I finish up a few other pending things on the tooling side).

1 Like
#14

Indeed they do.

1 Like
#15

Great. I know that @pavel.kirienko might be a bit disappointed, so I don’t want to rush and start anything. It’ll likely take me some more time to finish my other work, but let me know if we want to move forward with something.

FYI: What first prompted me to suggest this (other than the somewhat high complexity of libuavcan) is the fact that UDRAL and the V1 application layer, including PnP and other things, may see higher adoption if adopters have a reliable, clean reference implementation they can use.

#16

@coder_kalyan I’ll just point out that once udral (or whatever it ends up called) is implemented in Px4 and Ardupilot then the vast majority of peripheral manufacturers will use px4 CANNode or Ardupilot AP_Periph as the framework for their device firmware. Whilst there’s some bloat in that approach, it’s vastly outweighed by maintenance and risk benefits. It’s always good to have reference implementations, but in the udral domain I doubt there’s much need.

#17

I’m not so sure, I think there’s a much bigger world of servos, ESCs, INS system, etc with existing mature code bases that we want to be interested in adding support. Having an absolute minimal portable implementation they can drop in place and use with an existing CAN driver might help in those cases.

3 Likes
#18

Exactly. In this case, the less opinionated and complex our APIs are, the better.

#19

I can only strongly encourage the usage of C++ for any libuavcan_v1.

C and by extension C++ are the most portable languages by far. Of course you need to know how to use C++ within an embedded environment (which means: do not use any constructs that result in dynamic memory allocation at runtime. Dynamic allocation up-front, once, during initialisation is perfectly fine in my playbook), the same is valid for C. If you don’t access any parts of the C++ standard library (i.e. STL) than C++ should be as portable as C (although the ARM GCC has STL in its libstdcxx).

If you don’t know how to use them you’ll have problems either way. C++ allows a cleaner way to express common C idioms (ADT <-> class, static/dynamic polymorphism replaces writing your own vtable, …) and allows certain checks at compile time (static_assert, constexpr) which are simply not possible in C.

That being said you can still write a very nice C API and layer a C++ API on top. Case in point: 107-Arduino-UAVCAN.

Finally: I am really tired of the “language x is soooo flash/sram-hungry” argument … We are living in a time of incredible computing power available even in embedded devices. Hearing this always reminds me of “I hand code my assembly because my C compiler generates bad code.” I’ll trade ease of extension and maintenance by using good abstractions for a couple of bytes every time :wink: .

3 Likes
#20

Yeah, I’m not trying to make any arguments about the flash/resource usage of C or C++.

This may be a good middle ground as it does allow existing C projects to make use of the library, while still bringing benefits of C++ to newer projects.