Using Pyuavcan on Windows

pavel.kirienko · August 11, 2020, 9:34am

Can you please step through the reception pipeline with a debugger to see where your frames are being dropped? It makes sense to add some debug logging here as well.

polaris · August 11, 2020, 10:01am

Debug logging with “-vv” doesn’t show any INFO, DEBUG or ERROR messages during receiving these packets, at the end there are also no in_frames or in_frames_errored in the stats output. They are probably lost between can and media I guess.

P.S. Sorry, I found it, it was the handler of Media.ReceivedFramesHandler, trying to fix it

polaris · August 11, 2020, 11:32am

One more question, please.
Frames are processed properly now, uvc pub is working, uvc call has some problems with implementation:

uvc call 42 uavcan.node.GetInfo.1.0 “{}” --tr=“CAN(can.media.pythoncan.PythonCANMedia(‘pcan’,‘PCAN_USBBUS1’,8),10)”

2020-08-11 13:28:33 8460 ERROR pyuavcan._cli.commands.call: The transport implementation is misbehaving: feedback was never emitted; falling back to software timestamping. Please submit a bug report. Involved instances: client=Client(dtype=‘uavcan.node.GetInfo.1.0’, input_transport_session=CANInputSession(InputSessionSpecifier(data_specifier=ServiceDataSpecifier(service_id=430, role=<Role.RESPONSE: 2>), remote_node_id=42), PayloadMetadata(data_type_hash=0x6666666630239d01, max_size_bytes=313))), result=(uavcan.node.GetInfo.Response.1.0(protocol_version=uavcan.node.Version.1.0(major=1, minor=0), hardware_version=uavcan.node.Version.1.0(major=1, minor=0), software_version=uavcan.node.Version.1.0(major=0, minor=1), software_vcs_revision_id=0, unique_id=[1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0], name=‘our-servo’, software_image_crc=[], certificate_of_authenticity=’’), TransferFrom(timestamp=Timestamp(system_ns=1597145313213994200, monotonic_ns=21869359000000), priority=<Priority.NOMINAL: 4>, transfer_id=2, fragmented_payload=[<memory at 0x0000021AD1A2E048>, <memory at 0x0000021AD1A2E108>, <memory at 0x0000021AD1A2E1C8>, <memory at 0x0000021AD1A2E288>, <memory at 0x0000021AD1A2E348>, <memory at 0x0000021AD1A2E408>], source_node_id=42))

Where have I to search for a problem? I don’t see any dependencies to on_transfer_feedback in my python-can implementation.

pavel.kirienko · August 11, 2020, 11:36am

The library requested the transport to report when the transfer is delivered to the underlying network driver, but the report never arrived. Therefore, the transport is misbehaving. The error comes from this place specifically:

github.com

UAVCAN/pyuavcan/blob/1d5dbf07b6acc64576ca4f1deaae3b35bd7df377/pyuavcan/_cli/commands/call.py#L152


async def _run(client:        pyuavcan.presentation.Client[_S],
               request:       pyuavcan.dsdl.CompositeObject,
               formatter:     _subsystems.formatter.Formatter,
               with_metadata: bool) -> int:
    request_ts_transport: typing.Optional[pyuavcan.transport.Timestamp] = None


    def on_transfer_feedback(fb: pyuavcan.transport.Feedback) -> None:
        nonlocal request_ts_transport
        request_ts_transport = fb.first_frame_transmission_timestamp


    client.output_transport_session.enable_feedback(on_transfer_feedback)


    request_ts_application = pyuavcan.transport.Timestamp.now()
    result = await client.call(request)
    response_ts_application = pyuavcan.transport.Timestamp.now()


    # Print the results.
    if result is None:
        print(f'The request has timed out after {client.response_timeout:0.1f} seconds', file=sys.stderr)
        return 1
    else:

This is where the caller requested the transport to invoke the transmission feedback handler. Your implementation does not appear to handle it correctly.

polaris · August 11, 2020, 12:57pm

Thank you for your reply. I have already seen this part of code, I can’t just bind it with media layer.

I have found loopback parameter in _parse_native_frame function only, it is set if own transmitted packet was received successfully.

But I can’t find any connection between client.output_transport_session.enable_feedback(on_transfer_feedback)
and pyuavcan.transport.can.media.DataFrame:
https://github.com/UAVCAN/pyuavcan/blob/1d5dbf07b6acc64576ca4f1deaae3b35bd7df377/pyuavcan/transport/can/media/_frame.py#L19

I think it is the last thing I need to get the implementation ready. Actually I can work with a current version already, if you don’t need python-can, then I will stop annoying you.

pavel.kirienko · August 11, 2020, 1:23pm

The feedback flag is passed down the transfer serializer here:

github.com

UAVCAN/pyuavcan/blob/1d5dbf07b6acc64576ca4f1deaae3b35bd7df377/pyuavcan/transport/can/_session/_output.py#L126


                     transfer:           pyuavcan.transport.Transfer,
                     monotonic_deadline: float) -> bool:
self._raise_if_closed()


# Decompose the outgoing transfer into individual CAN frames
frames, auxiliary_iter = itertools.tee(_transfer_sender.serialize_transfer(
    compiled_identifier=can_id.compile(transfer.fragmented_payload),
    transfer_id=transfer.transfer_id,
    fragmented_payload=transfer.fragmented_payload,
    max_frame_payload_bytes=self._transport.protocol_parameters.mtu,
    loopback_first_frame=self._feedback_handler is not None
))
first_frame = next(auxiliary_iter)
num_frames = 1 + sum(1 for _ in auxiliary_iter)
assert num_frames > 0
del auxiliary_iter


# Ensure we're not trying to emit a multi-frame anonymous transfer - that's illegal
if can_id.source_node_id is None and num_frames > 1:
    raise pyuavcan.transport.OperationNotDefinedForAnonymousNodeError(
        f'Anonymous nodes cannot emit multi-frame transfers. CANID: {can_id}, transfer: {transfer}')

The serializer returns a generator of UAVCANFrame, where the first frame has the loopback flag set.

Upon seeing a frame where loopback is true/false, the SocketCAN driver enables/disables the corresponding socket option:

github.com

UAVCAN/pyuavcan/blob/1d5dbf07b6acc64576ca4f1deaae3b35bd7df377/pyuavcan/transport/can/media/socketcan/_socketcan.py#L219-L222


def _set_loopback_enabled(self, enable: bool) -> None:
    if enable != self._loopback_enabled:
        self._sock.setsockopt(socket.SOL_CAN_RAW, socket.CAN_RAW_RECV_OWN_MSGS, int(enable))
        self._loopback_enabled = enable

You need to recreate the loopback enable/disable logic in your media driver.

polaris · August 12, 2020, 1:44pm

Hello Pavel, I couldn’t implement it really tidy, python-can probably doesn’t receive transmitted messages, so I have just pushed a loopback message to an array and then place it to a normal receive buffer. Now everything works perfectly with Windows.

Should I commit my implementation or is it not good enough?

pavel.kirienko · August 12, 2020, 4:07pm

Send a pull request, please, we’ll give it a look. Also, I think it would benefit everyone if you informed the maintainers of Python-CAN about the need for supporting loopback frames. Maybe you could just open a ticket or something?

Thanks!

pavel.kirienko · December 8, 2020, 1:05pm

Incidentally, I just found out that Windows shell does not support single-quoted string literals. The only valid syntax is double-quote: "{}", whereas '{}' is parsed differently. I don’t really use Windows (other than in CI pipelines) so I wasn’t aware of this until now.