Development setup pycyphal (Ubuntu)

[maksim.drachov]

This guide assumes you’re using Ubuntu (in my case a VM running on Amazon EC2 as I’m on a Macbook).

1. Setup VM

If you already have a Ubuntu setup, you can skip this section.

1. Go to the EC2 Management Console
2. Launch Instance, with the following settings:
   • Name: pycyphal-vm
   • OS: Ubuntu
   • Instance type: t2.large (default t2.micro doesn’t have enough RAM)
   • Key pair: Create new key pair and save.
   • Other settings: default
3. Connect to instance using SSH client (using key pair)
   
   
   
   
   

Setup GitHub

1. Generate SSH key

   $ ssh-keygen -t ed25519 -C "your_email@example.com"
   

2. Add to agent

   $ eval "$(ssh-agent -s)"
   $ ssh-add ~/.ssh/id_ed25519
   

3. Add public key to GitHub settings:

   cat ~/.ssh/id_ed25519.pub
   

2. Setup Ubuntu

1. Install ncat, pip, nox

   sudo apt-get update
   sudo apt-get -y install ncat
   sudo apt install python3-pip
   sudo apt install nox
   

2. On EC2 VMs, the following errors might appear:

   subprocess`.CalledProcessError: Command '('sudo', 'modprobe', 'vcan')' returned non-zero exit status 1.
   

   To prevent this type of error:

   sudo apt-get install -y linux-modules-extra-$(uname -r)
   

   (source)

3. Setup pycyphal

1. Clone repo

   git clone git@github.com:OpenCyphal/pycyphal.git
   cd ~/pycyphal
   git submodule update --init --recursive
   

2. Run nox (this will setup a virtual environment that can then be used for pytest)

   nox --list  # Lists all available sessions
   nox --session test-3.10  # Runs the tests in Python 3.10
   # nox --session check_style  # Runs black code style check
   

3. Setup (necessary for pytest, recommended to save these commands in setup.sh to save some time!)

   export CYPHAL_PATH="/home/ubuntu/pycyphal/demo/custom_data_types:/home/ubuntu/pycyphal/demo/public_regulated_data_types"  # EC2 default username is ubuntu, replace if relevant ($HOME causes issues with Vscode debug)
   export PYTHONASYNCIODEBUG=1
   source .nox/test-3-10/bin/activate
   # source setup.sh # to save time
   

4. In another terminal window (leave open/running):

   ncat --broker --listen -p 50905
   

5. Pytest

   pytest pycyphal/  # Runs all unittest/doctests in pycyphal folder (code)
   pytest tests/     # Runs all unittests in tests folder
   pytest tests/transport/udp -k _output_session  # Runs only unittest from tests/transport/udp, whose unittest name matches the given substring (-k option)
   pytest tests/ --pdb # the --pdb option will stop execution upon failing a unittest and open a debug console
   

4. Setup Vscode

4.1 Remote Session

If you want to use Vscode, you’ll need to do the following steps:

1. Install the Remote SSH extension
2. Open Command Palette:
   1. Remote SSH: Connect to Host
   2. Configure SSH Hosts...
   3. /users/maksimdrachov/.ssh/config
   4. Add the new host: (use ssh command copied from EC2)
      
3. Open Command Palette:
   1. Remote SSH: Connect to Host
   2. Select added host
4. Open pycyphal folder
   
   

4.2 Debug

1. Install Python extension
2. Open Command Palette:
   1. Python: Configure Tests...
   2. pytest
   3. Use existing config file setup.cfg
3. For pytest to work correctly
   • Create file .env-variables with the following text:

   CYPHAL_PATH="/home/ubuntu/pycyphal/demo/custom_data_types:/home/ubuntu/pycyphal/demo/public_regulated_data_types"
   PYTHONASYNCIODEBUG=1
   

   (In my case username is ubuntu, this might be different for you)
   • Edit .vscode/settings.json:

   {
    "python.testing.pytestArgs": [],         # This should already be in place after step 2
    "python.testing.unittestEnabled": false, # and this
    "python.testing.pytestEnabled": true,    # and this
    "python.envFile": "${workspaceFolder}/.env-variables", # but not this one
   }
   

4. Restart Vscode window
   
   

Now you should be able to run unittests in Debug mode, place breakpoints,…

PS: In my case it is was put automatically, however make sure the Python interpreter from the nox session is used. If this isn’t the case:

• Open Command Palette
  • Python: Select Interpreter
  • Enter interpreter path...
  • Enter path: ./.nox/test-3-10/bin/python

5. Finishing touches

To leave the code as clean as you found it, the following tools are part of CI and so should be consulted before wrapping up a MR.

5.1 Black: code formatting

Install:

sudo apt install black

To run:

cd ~/pycyphal
nox --session check_style

Let’s say you did something wrong, now you can fix it:

(or you could just install the Black Formatter extension)

5.2 Pylint: code analysis

Install:

sudo apt install pylint

For example, you’re working on the udp part, run this:

pylint --disable=import-error,line-too-long pycyphal/transport/udp

(import-error and line-too-long are covered by MyPy and Black)

5.3 MyPy: static typing

Install:

sudo apt install mypy

For example, you’re working on the udp part, run this:

mypy pycyphal/transport/udp | grep pycyphal/transport/udp

(There’s probably a more conscise command? @pavel.kirienko)