HDF5 Event File Format

HDF5 is a standard hierarchical self-describing file format ideal to store datasets. Prophesee provides a lossless compression codec for event-based data called ECF (Event Compression Format) that is used in Metavision SDK to support our HDF5 event format.

You can download some HDF5 event files in our Sample Recordings page, or convert one of your RAW file into HDF5 event format using File to HDF5 sample.

Those HDF5 event files can be played with our applications (e.g. Metavision Studio) and our samples (e.g. Metavision Viewer).

In the SDK, HDF5 format is handled in Camera class (Metavision::Camera) in SDK Driver in C++ along with the RAW file format. Our HDF5 files can also be directly read and written using Metavision::HDF5EventFileReader and Metavision::HDF5EventFileWriter.

If you want to access the data of our HDF5 event files, you must use our ECF codec. It is delivered with Metavision SDK on both Ubuntu and Windows. It is also available as a shared library and an ECF filter plugin for HDF5 (with self-assigned filter code 0x8ECF) in the hdf5_ecf GitHub repository.

Using HDF5 files outside of Metavision SDK

The HDF5 tools and API can be directly used on our HDF5 event files. The bindings of HDF5 in other languages such as h5py can also be used to manipulate the HDF5 event files produced by Metavision SDK.

HDF5 must be made aware of the ECF codec to correctly interpret the data contained in the files. To do so, the HDF5_PLUGIN_PATH environment variable must point to the location of the ECF codec plugin. This can be accomplished by using the handy setup_env script (if working from the build folder) or modifying your environment (refer to the exact instructions for your platform and software deliverable in the installation guides).

Note

Using custom HDF5 filter plugins is slower than using directly the Metavision SDK. This should be reserved for quick prototyping or applications where the performance is not important.

Example using hdfview

Here, we use hdfview to show the list of attributes of our driving sample HDF5 event file:

And here, we look at the structure of the CD events stored in the file (x,y,p,t as explained in the event-based concepts page)

Example using h5dump

You can quickly extract information from one of our recording using h5dump. For example, on Ubuntu, we can extract the resolution (via the geometry attribute) of the turning spinner sample recording:

h5dump -a geometry spinner.hdf5

which would give the following ouput :

HDF5 "spinner.hdf5" {
ATTRIBUTE "geometry" {
  DATATYPE  H5T_STRING {
      STRSIZE H5T_VARIABLE;
      STRPAD H5T_STR_NULLTERM;
      CSET H5T_CSET_ASCII;
      CTYPE H5T_C_S1;
  }
  DATASPACE  SCALAR
  DATA {
  (0): "640x480"
  }
}
}

Or display the value of the first 3 events :

h5dump -d "/CD/events" -k 3 spinner.hdf5

which would output :

HDF5 "spinner.hdf5" {
DATASET "/CD/events" {
  DATATYPE  H5T_COMPOUND {
      H5T_STD_U16LE "x";
      H5T_STD_U16LE "y";
      H5T_STD_I16LE "p";
      H5T_STD_I64LE "t";
  }
  DATASPACE  SIMPLE { ( 54165303 ) / ( H5S_UNLIMITED ) }
  SUBSET {
      START ( 0 );
      STRIDE ( 1 );
      COUNT ( 1 );
      BLOCK ( 3 );
      DATA {
      (0): {
            237,
            121,
            1,
            0
        },
      (1): {
            246,
            121,
            1,
            0
        },
      (2): {
            248,
            132,
            1,
            0
        }
      }
  }
}
}

Note

To use HDF5 tools on Ubuntu, you may need to install an additional package :

sudo apt -y install hdf5-tools

Example using h5py

As a toy example, we want to count how many events occurred between two timestamps in a specified ROI :

import h5py
with h5py.File("spinner.hdf5", "r") as f:
  evts = f["CD"]["events"]
  print(len(evts[(evts['t']<49330) & (evts['t']>48270) & (evts['x']<300) & (evts['y']>300)]))

which would output :

33

Note

This is a simple and quite inefficient way of extracting information from a HDF5 recording. For better approaches, read the h5py documentation or use the Metavision SDK.