Tracking

The Analytics API provides two algorithms for object tracking:

• Metavision::TrackingAlgorithm which is a generic algorithm that tracks any moving object (by default)

  and can be tuned for more specific applications

• Metavision::SpatterTrackerAlgorithm which is a lighter implementation allowing to track simple non colliding moving objects

Each algorithm has a corresponding sample showing how to use it:

• <install-prefix>/share/metavision/sdk/analytics/samples/metavision_spatter_tracking

• <install-prefix>/share/metavision/sdk/analytics/samples/metavision_generic_tracking

Expected Output

Metavision Tracking samples visualize events and output bounding boxes around the tracked objects with an ID of the tracked object shown next to the bounding box:

Example of running Metavision Generic Tracking sample on the dataset file:

Example of running Metavision Spatter Tracking sample on the dataset file:

Setup & requirements

By default, Metavision Tracking looks for objects of at least 10x10 pixels size.

How to start

Here, we take Metavision Tracking sample as an example, however, Metavision Spatter Tracker runs in a similar way.

First, compile the sample as described in this tutorial.

To start the sample based on the live stream from your camera, run:

Linux

./metavision_generic_tracking

Windows

metavision_generic_tracking.exe

To start the sample based on recorded data, provide the full path to a RAW file (here, we use the file from Metavision Dataset):

Linux

./metavision_generic_tracking -i traffic_monitoring.raw

Windows

metavision_generic_tracking.exe -i traffic_monitoring.raw

To check for additional options:

Linux

./metavision_generic_tracking -h

Windows

metavision_generic_tracking.exe -h

Code Overview

Pipeline

Both samples implement the same pipeline:

Tracking Algorithm

The tracking algorithms consume CD events and produce tracking results (i.e. Metavision::EventSpatterCluster or Metavision::EventTrackingData). Those tracking results contain the bounding boxes with unique IDs.

These algorithms are implemented in an asynchronous way and process time slices of fixed duration. This means that, depending on the duration of the input time slices of events, the algorithms might produce 0, 1 or N buffer(s) of tracking results.

Like any other asynchronous algorithm we need to specify the callback that will be called to retrieve the tracking results when a time slice has been processed:

// Sets the callback to process the output of the spatter tracker
tracker_->set_callback(
    [this](const Metavision::timestamp ts, const std::vector<Metavision::EventSpatterCluster> &clusters) {
        tracker_callback(ts, clusters);
    });

Frame Generation

At this step, we generate an image where the bounding boxes and IDs of the tracked objects are displayed on top of the events.

Here, we are not using the Metavision::Pipeline utility class to implement the pipeline. As a consequence, to ease the synchronization between the events and the tracking results, the Metavision::EventsFrameGeneratorAlgorithm class is used. This class allows us to buffer the input events (i.e. Metavision::EventsFrameGeneratorAlgorithm::process_events()) and to generate the image on demand (i.e. Metavision::EventsFrameGeneratorAlgorithm::generate_image_from_events()). After the event image is generated, the bounding boxes and IDs are rendered using the Metavision::draw_tracking_results() function.

As the output images are generated at the same frequency as the buffers of tracking results produced by the tracking algorithms, the image generation is done in the tracking algorithms’ output callbacks:

void Pipeline::tracker_callback(Metavision::timestamp ts,
                                const std::vector<Metavision::EventSpatterCluster> &trackers) {
    if (tracker_logger_)
        tracker_logger_->log_output(ts, trackers);

    if (frame_generator_) {
        frame_generator_->generate_image_from_events(ts, back_img_);

        Metavision::draw_tracking_results(ts, trackers.cbegin(), trackers.cend(), back_img_);
    }

    if (video_writer_)
        video_writer_->write_frame(ts, back_img_);

    if (displayer_)
        displayer_->swap_frame(back_img_);

    if (timer_)
        timer_->update_timer(ts);
}

while the buffering of the events is done in the Metavision::Camera’s output callback:

// Connects filters
camera_->cd().add_callback([this](const Metavision::EventCD *begin, const Metavision::EventCD *end) {
    // Frame generator must be called first
    if (frame_generator_)
        frame_generator_->process_events(begin, end);

    tracker_->process_events(begin, end);
});

Note

Different approaches could be considered for more advanced applications.

Display

Finally, the generated frame is displayed on the screen, the following image shows an example of output: