Data Encoding Formats

Prophesee event-based sensors are producing data in two different types of proprietary formats:

  • Event Stream Data Format: EVT 2.0, EVT 2.1 and EVT 3.0 (where EVT stands for EVent Type)

  • Event Frame Data Format: Histo3D and Diff3D (where “frames” are computed from events)

Those event formats have been designed to strike a balance between compactness and robustness, while taking into account constraints imposed by the processing platforms.

See also

For an overview on event stream data formats, you can watch our training video:

`

Event Stream Data Formats

Event Stream Data formats are the most native formats for our event-based sensor, as they transmit the raw events detected by the pixels (timestamp, coordinates and polarity). The following schemes of encoding can be used:

  • EVT 2.0: 32-bit non-vectorized format for low event rate

  • EVT 2.1: 64-bit vectorized format for high event rate

  • EVT 3.0: 16-bit vectorized and more compressed format for high event rate

The availability of the formats depends on the sensor:

  • IMX636 sensor is supporting EVT 2.1 and EVT 3.0 (default)

  • GenX320 sensor is supporting EVT 2.0, EVT 2.1 (default) and EVT 3.0 (only on the MP version of the sensor, not on the ES version)

Event Frame Data Formats

Those formats are producing frames of data made of arrays of values representing the number of pixels having spiked during an integration period or until reaching an event rate threshold.

Those formats accumulate event-data received during an integration period or until reaching an event rate threshold, and represent it in framed form. These formats enable avoiding the cost of decoding, pre-processing and formatting the event stream in software, and are therefore particularly useful for light-weight computing platforms.

The following formats are available:

  • Histo3D: accumulation of event spikes per pixel/polarity

  • Diff3D: increment/decrement according to polarity of event spikes per pixel

Other Data Formats

GenX320 sensor support also AER (Address Event Representation) format

Note

GenX320 supports AER format through the CMOS Parallel Interface (CPI) only. It is mainly destined for integration with neuromorphic systems.

Selecting Data Format

When you are streaming from a live camera with a sensor that supports multiple data formats using a camera plugin that implemented those format, you can choose which format you want to stream in.

The first and easiest option is available in Metavision Studio in the advanced panel when opening a camera. Here is an example using an EVK4 HD that comes with an IMX636 sensor supporting both EVT 2.1 and EVT 3.0:

Choosing data format in Studio

You can also select the data format through the SDK API using Metavision::DeviceConfig class. The following code excerpts show how to select the EVT 2.1 format:

  • Using HAL C++ API:

    Metavision::DeviceConfig device_config;
device_config.set_format("EVT21");
device = Metavision::DeviceDiscovery::open(serial, device_config);

  • Using HAL and SDK Core Python API:

    device_config = DeviceConfig()
device_config.set_format("EVT21")
device = DeviceDiscovery.open("", device_config)
mv_iterator = EventsIterator.from_device(device=device)

  • Using SDK Stream C++ API (Camera class):

    Metavision::DeviceConfig device_config;
device_config.set_format("EVT21");
camera = Metavision::camera::from_first_available(device_config);