Data Encoding Formats

Prophesee’s event-based sensor are producing data in two different types of proprietary formats:

Event Stream Data Format: EVT 2.0, EVT 2.1 and EVT 3.0 (where EVT stands for EVent Type)

Event Frame Data Format: Histo3D and Diff3D (where “frames” are computed from events)

Those event formats have been designed to strike a balance between compactness and robustness, while taking into account constraints imposed by the processing platforms.

Event Stream Data Formats

Event Stream Data formats are the most native formats for our event-based sensor, as they transmit the raw events detected by the pixels (timestamp, coordinates and polarity). The following schemes of encoding can be used:

EVT 2.0: 32-bit non-vectorized format for low event rate

EVT 2.1: 64-bit vectorized format for high event rate

EVT 3.0: 16-bit vectorized and more compressed format for high event rate

The availability of the formats depends on the sensor:

IMX636 sensor is supporting EVT2.1 and EVT 3.0 (default)

GenX320 sensor is supporting EVT 2.1 only at the moment

Event Frame Data Formats

Those formats are producing frames of data made of arrays of values representing the number of pixels having spiked during an integration period or until reaching an event rate threshold.

Those formats accumulate event-data received during an integration period or until reaching an event rate threshold, and represent it in framed form. These formats enable avoiding the cost of decoding, pre-processing and formatting the event stream in software, and are therefore particularly useful for light-weight computing platforms.

The following formats are available:

Histo3D: accumulation of event spikes per pixel/polarity

Diff3D: increment/decrement according to polarity of event spikes per pixel

Other Data Formats

GenX320 sensor support also AER (Address Event Representation) format

Note

GenX320 supports AER format through the CMOS Parallel Interface (CPI) only. It is mainly destined for integration with neuromorphic systems.

Selecting Data Format

When you are streaming from a live camera with a sensor that supports multiple data formats using a camera plugin that implemented those format, you can choose which format you want to stream in.

The first and easiest option is available in Metavision Studio in the advanced panel when opening a camera. Here is an example using an EVK4 HD that comes with an IMX636 sensor supporting both EVT 2.1 and EVT 3.0:

You can also select the data format through the SDK API using Metavision::DeviceConfig class. The following code excerpts show how to select the EVT 2.1 format:

Using HAL C++ API:

Metavision::DeviceConfig device_config;
device_config.set_format("EVT21");
device = Metavision::DeviceDiscovery::open(serial, device_config);

Using HAL and SDK Core Python API:

device_config = DeviceConfig()
device_config.set_format("EVT21")
device = DeviceDiscovery.open("", device_config)
mv_iterator = EventsIterator.from_device(device=device)

Using SDK Driver C++ API (Camera class):

Metavision::DeviceConfig device_config;
device_config.set_format("EVT21");
camera = Metavision::camera::from_first_available(device_config);