Note
This Python sample is available only with our Professional plan.
Inference Pipeline of Optical Flow
The script allows you to quickly set up an inference pipeline for Optical Flow Estimation.
You can use it with the pretrained PyTorch flow model flow_model_alpha.ckpt
from our pre-trained models.
Expected Output
The script takes events or preprocessed event features as input and outputs optical flow.
An example of the output is shown below:
Setup & requirements
To run the script, you will need:
a pretrained PyTorch model (e.g.
flow_model_alpha.ckpt
from our pre-trained models)an event-based camera or a
RAW
,DAT
orHDF5
input file. You can find event-based files in our Sample Recordings page
Note
Since HDF5 contains preprocessed features, you need to be sure that the same preprocessing method is used for the flow model and the HDF5
file(s).
For instance, our trained flow model uses event_cube
method, so if you want to use HDF5 files as input, they need
to be processed with event_cube
as well.
How to start
To start the script based on recorded data, you need to provide the full path to the input file and the path to the pre-trained model. For example:
Linux
python3 flow_inference.py /path/to/flow_model_alpha.ckpt pedestrians.raw
Windows
python flow_inference.py /path/to/flow_model_alpha.ckpt pedestrians.raw
Note
To read directly from a camera, provide the camera serial number if there are several cameras otherwise leave it blank.
Use
-w /path/to/output.mp4
to generate a mp4 videoUse
--mode
to specify your visualization mode:arrows (default mode): the size of the arrow represents the flow magnitude, while the color of the flow represents the angle
sharp : use this mode to check the quality of flow estimation, if the input features contain micro time bins, in other words, if the preprocessing method is
event cube
ormulti_channel timesurface
. This will give sharper edges when the flow direction is correctly modelled.
Warning
To make the best use of the model, it is important to set the accumulation time interval (--delta-t
) correctly in corresponding to the speed motion of the object. It is especially important for objects with high-speed motions: a long accumulation time would produce erroneous results.
To find the full list of options, run:
Linux
python3 flow_inference.py -h
Windows
python flow_inference.py -h