Treuzell host-side implementation

Treuzell discovery

With Treuzell, cameras are seen as remote boards, and the first task of the host is to establish a connection with the board. This is also the role of Metavision HAL CameraDiscovery. TzCameraDiscovery is an implementation following HAL interface, based on libusb, to detect cameras exposing Treuzell interface over USB. This is currently the only existing implementation, but there could, in theory, exist several inside a plugin. For instance, there could be one using SSDP (Simple Service Discovery Protocol), like :UPnP (Universal Plug and Play), then establishing connection over TCP/IP. There could also be one looking for V4L2 devices, and then also implementing the device side part of Treuzell to convert commands to V4L2 system calls, allowing to keep the generic mapping between Treuzell and Metavision HAL provided by the plugin.

When a Treuzell-compatible camera is discovered, a handler is created, generically called a BoardCommand, to handle future communications with the camera, and then the device builder (a TzDeviceBuilder, not to be mistaken with USB devices or HAL device) is asked if it knows a driver for each Treuzell device on the board, and, if so, the board is advertised to HAL with its serial number.

Treuzell LibUSB discovery

The implementation over libusb simply lists every device over USB, and then, for the known vid (Vendor IDentifier)/pid (Product IDentifier), looks for a USB interface with Vendor Class, Treuzell sub-class, with the right endpoints. The current implementation is pedantic regarding the interface composition and will simply ignore a board displaying endpoints in a different order or using different endpoint types.

There is however a trick regarding vid/pid: different products usually use different vid:pid, but Treuzell is not a standard USB class, thus it goes into Vendor class, and each Vendor manages its vendor-classes numbering on its own. Hence we need to register each implementer with the USB subclass used for Treuzell.

Treuzell board command

A BoardCommand is an internal object in Prophesee plugin implementation to keep track of communication information. Currently, for Treuzell, there is a single implementation, called TzLibUSBBoardCommand, over libusb. There should be one implementation per communication bus, or way to access the bus (as USB is manageable without libusb), working in pair with a CameraDiscovery implementation.

The purpose of the BoardCommand is to abstract everything communication-related, and just provide methods to transfer Treuzell control frames. For convenience, it also provides methods to get some information on the board, such as its name, serial, the number of devices… As those are just generic Treuzell board properties, this could be implemented in a generic parent TzBoardCommand class if other implementations are created.

Treuzell LibUSB board command

Treuzell board command for boards addressed through USB using libusb is called TzLibUSBBoardCommand. As expected from a board command implementation, it implements Treuzell commands transfer. Part of this task is to keep track of libusb internal handlers for its life duration, and free resources at destruction. As libusb is a C library, part of it was wrapped into C++ objects, to manage duration with smart pointers. The purpose of this wrapper is not to be comprehensive, just to handle generically a few selected elements from libusb. Especially, as its use was pushed back in some legacy implementation, the DataTransfer can be and is shared with the Fx3 code. The DataTransfer class is responsible for transferring a high-bandwidth raw stream over USB, in our case the sensor’s events stream, and is protocol-agnostic (besides USB, managed through libusb).

Treuzell Device builder

Once the board command is built, its first use is for the discovery to get information regarding the board hardware, to know if the running plugin can manage it. The way to do so is to request the number of Treuzell devices on the board, and for each device, what it is compatible with. This last part relies on compatibility strings, as used by OpenFirmware device tree. The main idea is to have a base driver as a fallback when the device is not known, allowing to have the driver logic either/both on host or/and board side, with limited functionalities when an ad-hoc driver is not found.

In the context of multiple plugins, there is no fallback, and the build of a board command will fail if one of the devices on the board is not known. Then the board is not reported by the discovery, and HAL can ask to the next plugin to run its own discovery, until the process reaches the plugin designed for this board.

Moreover, in some cases, the same device exists on different boards, and the plugin may register an extra check method along with the build method, to restrict cases in which the board is advertised through discovery. This, however, should not be the base choice for a design and was mostly introduced to create incompatibility when the board was already released. If a Treuzell device is an implementation with some specific extensions, it is best to advertise several compatibility strings, from the most specific to the most generic description of the device, allowing the host to choose the driver that can best use the device.

The device builder is instantiated by the discovery during its construction, and the plugin can register the build methods for the known devices during initialization. This implementation is a bit heavy to maintain, and it would be more convenient to automatically register build methods and compatibility strings based on what is built and linked, since the compiler’s directives already exhaustively describe the content of each plugin.

Treuzell Device

Previous sections mentioned Treuzell devices without fully explaining what it is. In Treuzell’s model, a camera is a board with several devices on it. A Treuzell device may be any part reusable from one board to another, such as the sensor. The devices on a board are assumed to form a pipeline with, for instance, the sensor passing its data to the chip doing a bridge from camera interface to USB, and then this chip sending data on USB.

The default assumption is that the devices are pipelined in the order of their index on the board, with the lowest index (0) being the device that sends data on the communication link. It is however possible to add some commands in Treuzell to describe the devices topology, assuming it is the default one if the command is not implemented by the board.

The basic commands for a device are enable/disable and stream on/off, using the property names from Treuzell. However, part of the implementation uses another terminology:

• init for enable

• destroy for disable

• start for stream on

• stop for stream off

This terminology forms what we call the ISSD (Init, Start, Stop, Destroy) describing the complete sequence for a single run.

Both terminologies will be used in this documentation, with no semantic difference.

• Enable: brings a device to a reasonable state for streaming. Usually: starts power supply, allocates memory, sets a configuration.

• Stream on: starts streaming with the current configuration.

• Stream off: stops streaming and comes back to a state ready to start a new stream. This usually includes clearing the internal pipeline memories, if any, to avoid starting the new stream with remaining data from the previous run. This is not an absolute rule however: if the source is not live, for instance if it is a fixed pattern generator, or a recording, the need a bit-accurate output, with no drop, may take precedence over this general rule.

• Disable: puts the device back to its original state, hence gracefully stops internal units, releases allocated memory, disables power supply.

Some devices may not actually be on the data streaming path but be exposed through Treuzell to provide some out-of-band information on the board, such as temperature. Those devices should simply ignore streaming commands.

In this implementation, there is directly a TzDevice class, instantiated (usually a derived class) for each Treuzell device, handling this sequence:

• Enable when the object is created

• Disable when the object is destroyed

• Stream on/off handled through start/stop methods

The historical approach at Prophesee was to put all the intelligence on host side, as users were more used changing code on their computer than on embedded targets. On most devices, ISSD (Init, Start, Stop, Destroy) methods are implemented through register accesses. This approach, however, cannot apply to more complex systems, with power management and dynamic memory allocation, a nd the trend is to generalize the use of Enable and Stream properties from Treuzell, and push at least part of the intelligence to the board.

A complete implementation also allows some configuration customization, to do between Init and Start, and there is currently no method to advertise most of those options in Treuzell, thus everything is handled with register accesses from the host. On a system with several execution levels, it is usually not allowed to an unprivileged user to make direct register accesses, allowing this from a remote host is even more dubious from a security point of view. At some point, custom control on devices should therefore be considered at Treuzell level.

This drives to a generic implementation on host side, pushing any specific knowledge of the system to the board in the nominal case, reserving direct register accesses for development, for instance on test boards. While custom controls are optional for a first implementation, it is mandatory to advertise at least the output data format of the system to the host to get to a generic TzDevice.

Treuzell Device with regmap

As traditionally, most of the device management is handled with register accesses, it is more convenient to access registers by name, with a class managing the mapping to an address on the current device/board. This class is called RegisterMap in Prophesee plugins, and uses some lambda functions to generate actual accesses on Treuzell board command.

In Treuzell implementation, there is a TzDeviceWithRegmap class that manages regmap instantiation and lambdas generation to map accesses to the right Treuzell device when accessing the board command.

Treuzell Device with ISSD sequences

The start-up sequence of a sensor may be complex and reference sequence is provided as a list of register accesses. Those register sequences are managed by the TzIssdDevice class.

As the RegisterMap is shared by multiple board commands, the register access logging is done there. This led to make TzIssdDevice derive from TzDeviceWithRegmap, even if the regmap is not used when applying ISSD (Init, Start, Stop, Destroy) sequences.