New OpenCyphal Project Proposal: CETL

Development & Maintenance
#1

As we develop libcyphal and as C++ Nunavut matures there is an emerging need to share some basic container types between the two projects that are either not readily available in STL or not available in STL at our base support level of C++14. As such we have three ways forward:

  1. Write duplicate types for both projects and share nothing. This is a bad plan since it forces the user to convert between two thread-bare implementations or co-ordinate overrides to their preferred implementations between both projects.

  2. Use Nunavut types in libcyphal directly. This seems odd and problematic since the Nunavut types don’t exist until nnvg generation has completed and for reasons discussed in the thread below…

  3. Start a new OpenCyphal project: Cyphal Embedded Template Library. This would be a common dependency of all OpenCyphal C++ code but would remain a disciplined set of types that are in active use by OpenCyphal projects.

CETL
CETL2740Ă—1080 163 KB

Project Tenets

  • CETL does not supplant STL, ETL, boost, or any other full-featured support library. It is not a general-purpose C++ support library and is limited to the minimum set of types needed to ensure OpenCyphal C++ projects are agnostic to these larger projects.
  • All OpenCyphal projects that use CETL shall allow total replacement of CETL types by providing compile-time overrides.
  • CETL types should support replacement by C++17 and newer concepts. Where a popular library diverges from the C++ standard CETL will prefer the standard (e.g. if ETL’s span and the C++20 span have different interfaces or behaviors CETL will mimic C++20)
  • CETL types will never require use of the default STL heap. They may require the user to specify an allocator before they can be instantiated.

To help clarify, this is how the initial set of dependencies would work:

CETL dependencies
CETL dependencies638Ă—570 12.6 KB

I’m sure this list of tenets will grow as we discuss the proposal. Your feedback is requested:

@pavel.kirienko , @erik.rainey, @aeriq, @lydiagh

1 Like
#2

Yet another C++ container library (YTL), yay! The tenets look sensible, but look:

  1. Use Nunavut types in libcyphal directly. This seems odd and problematic since the Nunavut types don’t exist until nnvg generation has completed. This would make static analysis of the libcyphal codebase pretty hard.

One of the basic tenets for libcyphal was to provide a batteries-included experience compared to the bare-PCB-solder-your-own-power-supply experience offered by libxxxard. Do I get this correctly? One consequence, if so, is that at least the high-level part of libcyphal will inevitably depend on DSDL-generated types. Such as, for instance, the register types, which I mentioned recently on Matrix; or, perhaps, the uavcan.file.* types, or diagnostics, and so on. This is already the case for PyCyphal where we also do static analysis, which works by treating DSDL compilation as part of the normal build process. If we accept that building libcyphal requires building DSDL first (which I think is a requirement for any Cyphal application anyway), then the dependency of libcyphal on nunavut-generated support types does not seem like a design flaw.

wdyt?

#3

This was my initial thought too but there’s an impedance mismatch in relying on Nunavut types for application-layer functions and relying on it for transport or media layer types.

#4

Indeed, but even if you attempt to match the impedance inside libcyphal, at one point between the application layer and the transport layer you will have to depend on DSDL-generated types anyway, even if they are not directly exposed to the user (which I think is admissible in at least some cases, but this is a topic for another debate).

#5

One of the design tenets of libcyphal is that we can bring up the transport layer without the application layer. It seems to me this means that layer should be independent of any specific DSDL type and usable without nnvg codegen.

#6

Are we sure that this:

One of the design tenets of libcyphal is that we can bring up the transport layer without the application layer.

Implies this:

that layer (should be independent of) <…> and (usable without) nnvg codegen.

But not this:

that layer (should be independent of) <…> and (usable without) CETL.

Or, in other words, you don’t want to depend on Nunavut only for the sake of its support library and propose that maintaining a separate small library is the preferred alternative?

#7

I do not think that the transport layer should be independent of TransportLayer[CETL := UserOverride]

That’s correct.

#8

Okay. Do you already have a set of entities in mind that will be provided by the CETL API?

#9

the variable_length_array from nunavut (possibly renamed), a span type, and the nunavut variant. We might also want to include an allocator or two so we can use the same memory pool in the libcyphal transport layer for (de)serializing types…perhaps? Definitely just start with the first three though.

#10

image
image680Ă—875 60 KB

#11

Looks like expected, unexpected, too. Are those staying?

#12

Does the “okaaaay” face mean you consider CETL a failure of design?

#13

It is my seal of endorsement.

#14

This is my seal of approval:

image

Looks good to me!

1 Like
#15

I’m feeling/have felt the pain of not having such a thing as CETL and therefore highly endorse it :wink: .

#16

Should we name the types that are drop-in replaceable with std:: alternatives the same as they are named in std to enhance compatibility? This would mean that the cetl::variable_length_array would become cetl::vector.

#17

The types could be named similarly but don’t need to be. The API however will need to be very similar so that the std:: types can be dropped in easily with clever usings when customers have that available.

#18

Perhaps “compatibility” is a poor choice of word. What I mean is that from naming alone it would be evident that cetl::vector<> and std::vector<> are meant to be (somewhat) compatible and (mostly) drop-in replaceable, but if the former was named cetl::variable_length_array<>, the clarity is gone.

#19

Yeah, I agree, where the type can be textually replaced. variable_length_array is an interesting case since it does things that vector cannot.

I disagree. I think the types should be textually replaceable save the change in namespace. For example, libcyphal could use CETL like this:

#if __cplusplus >= 202002L

#    include <span>
namespace libcyphal
{
namespace types
{
template <typename T>
using span = std::span<T>; 
}  // namespace types
}  // namespace libcyphal

#else
#    include "cetl/span.hpp"

namespace libcyphal
{
namespace types
{
template <typename T>
using span = cetl::span<T>; 
}  // namespace types
}  // namespace libcyphal

#endif
#20

We will have to establish some tenets and guidelines for naming since functionality is not 1:1. Naming things in a similar way communicates correct notions and incorrect notions.

Documentation is key here.

Some concepts just don’t carry over equivalently since common features are missing (no heap, no exceptions?). Vectors are possibly the most contentious name and the most useful object in this discussion. They are typically thought of as a dynamically sized array. It’s their key feature. If they aren’t dynamic in cetl anymore then are they just arrays? Do they map to something in between std::array and std::vector while not being the same as either?

Many of the objects would be 1:1 if they don’t have dynamic allocation in them, like your span example. I’m concerned that the std::vector drop-in would have to be an object with a templated size, which makes using it with a using statement somewhat hard. ETL has this exact problem.

That template definition of their vector is not a drop in replacement for std::vector, which has no size_t field. Fundamentally they aren’t compatible constructions. Obviously some APIs are not possible to implement either in 1:1 fashion like resize (which does not return a value on failure).

I guess what I’m saying is don’t make a vector type. It sets the wrong expectations.

1 Like