Optical Flow Sample

The Computer Vision API can be used to compute the optical flow of objects moving in front of the camera. The optical flow is computed in a sparse way: flow information is generated on clusters of events and not for each event.

The sample in <install-prefix>/share/metavision/sdk/cv/samples/metavision_sparse_optical_flow/metavision_sparse_optical_flow.cpp shows how to implement a pipeline for computing the sparse optical flow.

Expected Output

The sample visualizes events and the output optical flow with arrows indicating direction and magnitude of motion:

The sample also generates a video with the output flow.

How to start

First, compile the sample as described in this tutorial.

To start the sample, you need to provide recorded data with the full path to a RAW file (here, we use the file from Metavision Dataset):

Linux

./metavision_sparse_optical_flow pedestrians.raw

Windows

metavision_sparse_optical_flow.exe pedestrians.raw

Code Overview

Pipeline

Metavision Optical Flow sample implements the following pipeline:

Spatio Temporal Contrast Filter Stage

This stage applies the Metavision::SpatioTemporalContrastAlgorithmT to reduce the noise in the events stream.

The filtered events are then sent to both the optical flow and frame generation stages.

Sparse Optical Flow Stage

This stage applies the Metavision::SparseOpticalFlowAlgorithm on the events stream and produces Metavision::EventOpticalFlow events for each internally detected events cluster. Events are clustered together based on their speed and directions: events with a similar speed and direction are clustered and flow is computed for them.

Frame Generation Stage

This stage uses the Metavision::FlowFrameGeneratorAlgorithm to generate a frame that will be used later on in the display stage to visualize the result of the Metavision::SparseOpticalFlowAlgorithm by rendering the estimated flows on the top of the events.

As shown in the graph, the optical flow events and the CD events are received in parallel meaning that the optical flow events and CD events have to be synchronized in some ways. This is done in the Metavision::FlowFrameGenerationStage by buffering the incoming events and producing the image when possible.

Note

Different approaches could be considered for more advanced applications.

In the Metavision::FlowFrameGenerationStage class, the methods Metavision::BaseStage::set_previous_cd_stage() and Metavision::BaseStage::set_previous_flow_stage() allow to redirect the input data to the corresponding consuming callback.

When ready, the output frame is sent to the display stage.

Display Stage

This stage allows us to visualize the previously generated image on the screen: