Concept
When operating Cyphal tools that manage DSDL, the user is required to provide the DSDL sources manually. Examples of this include the use of Yakut, Yukon, or Nunavut. The tool UX could be improved if they were able to locate the required DSDL definitions automatically and the user could utilize the semantic versioning features of DSDL directly without the need to manage the DSDL source versions manually. This is easily implementable through a centralized repository of DSDL namespaces.
The workflow I envision is as follows: when a tool encounters an unknown DSDL data type name, it would automatically fetch the list of all known DSDL namespaces — not only standard but also vendor-specific — from some URI on the web, parse it, and check if the required root namespace is available there. If it is, it would automatically fetch the DSDL source archive from the URI associated with that namespace and process it.
This approach would, ideally, obviate the need for manual namespace management completely, as if all of the well-known DSDL namespaces in existence were always available to the tools locally. One might see how this approach mirrors the package repository design for some programming languages, such as PyPI or the Rust crate registry.
To enable vending of custom repositories, the user should be able to specify a custom URI for the repository. The default repository is to contain strictly only the public regulated namespaces.
A repository definition can inherit namespaces from other repositories. The public regulated repository is not allowed to depend on other repositories so as to prevent unintended inclusion of third-party namespaces and the formation of cyclic dependencies.
Proposed implementation
Define the entry schema for the repository in YAML; variables are shown in ALL_CAPS:
# List of repositories whose contents are entirely included into this one.
# Cyclic dependencies are not permitted.
extends: # Assume empty if missing.
- uri: REPOSITORY_URI
- ...
# List of namespaces defined in this repository.
# If a namespace defined here is also defined in any of the inherited repositories,
# the local definition overrides the inherited one.
namespaces: # Assume empty if missing.
ROOT_NAMESPACE_NAME:
source:
zip:
uri: ZIP_ARCHIVE_URI
path: PATH_INSIDE_ZIP_ARCHIVE
dependencies:
namespaces: [ROOT_NAMESPACE_NAME, ...]
...
Definition of the root repository https://dsdl.opencyphal.org/public_regulated.yaml:
# The root repository is not allowed to depend on any other repository.
# The root repository is only allowed to define public regulated namespaces.
namespaces:
uavcan:
source:
zip:
uri: https://github.com/OpenCyphal/public_regulated_data_types/archive/refs/heads/master.zip
path: /
dependencies:
namespaces: []
reg:
source:
zip:
uri: https://github.com/OpenCyphal/public_regulated_data_types/archive/refs/heads/master.zip
path: /
dependencies:
namespaces: [uavcan]
The root public_regulated.yaml repository file will be hosted in the existing public regulated data types GitHub repository.
Example of a vendor-specific repository, say, https://dsdl.zubax.com/repository.yaml:
extends:
- uri: https://dsdl.opencyphal.org/public_regulated.yaml
namespaces:
zubax:
source:
zip:
uri: https://github.com/Zubax/zubax_dsdl/archive/refs/heads/master.zip
path: /
dependencies:
namespaces: [uavcan]
Users are expected to define their own local repository.yaml file which normally should only contain links to external repositories and define none of the namespaces by itself:
# ~/.dsdl.yaml
extends:
- uri: https://dsdl.opencyphal.org/public_regulated.yaml
- uri: https://dsdl.zubax.com/repository.yaml
- ...
Cyphal tools would ingest the location of the repository file via the environment variable like CYPHAL_REPO, somewhat similar to CYPHAL_PATH; the default value should be https://dsdl.opencyphal.org/public_regulated.yaml.
Usage in tools
Yakut
When the tool encounters a command like this:
y pub -T 0.1 10:reg.udral.service.common.readiness 3
It would automatically detect the root namespace reg, fetch the repo file, fetch the archive, find the root namespace directory in it, compile it, and cache the compiled definitions locally. This will only be done once and the cached data will persist until the user references a data type version that is not present in the local cache, which will force the cache to be invalidated. Thus, the versioning capability built into DSDL will drive the cache invalidation automatically.
The “compile” command and the associated installation steps will thus be eliminated.
Yukon
When the tool encounters a type name register matching uavcan\.(pub|sub|cln|srv)\..*\.type that contains a data type name unknown to the tool (not present in the local cache), it would automatically consult with the registry and only alert the user of the unknown data type if it is not available in the registry.
Nunavut
While not a priority, I will list this possibility for completeness anyway. Currently, Nunavut is invoked with the specific DSDL root namespace path (or several):
nnvg --target-language c public_regulated_data_types/reg --lookup-dir public_regulated_data_types/uavcan
With this proposal implemented, one could do this instead; observe that it is no longer necessary to specify the lookup directories:
nnvg --target-language c --namespace=reg