Applications and Tools

Explore the capabilities of Metavision SDK by following one of these paths:

Discover the various applications that leverage our event-based optimized algorithms

Check out our ready-to-use tools available to work with event-based devices and recorded files

In addition, detailed information required to develop an optimal setup for your specific application using Prophesee technology is available on this KC page

Applications

Metavision SDK can be used for a variety of application fields:

Name	Description	Samples
Particle Size Monitoring	Control, count and measure the size of objects moving at very high speed in a channel or a conveyor. Get instantaneous quality statistics in your production line, to control your process.	metavision_psm in C++ metavision_psm in Python
Object Tracking	Track moving objects in the field of view. Leverage the low data-rate and sparse information provided by event-based sensors to track objects with low compute power.	metavision_generic_tracking in C++ metavision_generic_tracking in Python
Vibration Monitoring	Monitor vibration frequencies continuously, remotely, with pixel precision by tracking the temporal evolution of every pixel in a scene.	metavision_vibration_estimation in C++ metavision_vibration_estimation in Python
Spatter Tracking	Track small particles with spatter-like motion	metavision_spatter_tracking in C++ metavision_spatter_tracking in Python
High-Speed Counting	Count objects at unprecedented speeds, high accuracy, generating less data and without any motion blur.	metavision_counting in C++ metavision_counting in Python
Edgelet Tracking	Track 3D edges and/or Fiducial markers for your AR/VR application	metavision_model_3d_tracking in C++ metavision_model_3d_tracking in Python
Optical Flow	Understand motion through continuous pixel-by-pixel tracking and not sequential frame by frame analysis.	metavision_sparse_optical_flow in C++ metavision_dense_optical_flow in C++ metavision_sparse_optical_flow in Python metavision_dense_optical_flow in Python
Jet Monitoring	Monitor jets (i.e. dots) that are being dispensed	metavision_jet_monitoring in C++ metavision_jet_monitoring in Python
Active Marker 2D Tracking	Track Active Markers in 2D.	metavision_active_marker_2d_tracking in C++
Active Marker 3D Tracking	Track Active Markers in 3D.	metavision_active_marker_3d_tracking in C++
ArUco Marker Tracking	Track ArUco Markers	metavision_aruco_marker_tracking in C++

Note

For optimal results when using samples listed above, you should pay attention on event stream quality and/or background noise. To deal with that, hardware settings/filters (biases and hardware ESP) and software filters (demonstrated in samples Noise Filtering and Filtering) are available.

In the case of ESP note that AFK and STC functions are available both in hardware and software versions.

Metavision SDK can also manipulate event-based datasets and design event-based Neural Networks. With our pre-trained models, inference can be done with the following use cases:

Name	Description	Samples
Detection Inference	Leverage our pretrained automotive model written in pytorch, and experiment live detection & tracking	metavision_detection_and_tracking in C++ metavision_detection in C++ metavision_detection_and_tracking in Python
Optical Flow Inference	Predict optical flow from Event-Based data leveraging our pretrained Flow Model, customized data loader and collections of loss function and visualization tools	flow_inference in Python flow_inference in C++
Corner Detection & Tracking Inference	Detect and track corners in an event stream with high efficiency. This method can generate stable keypoints and very long tracks	demo_corner_detection in Python
Gesture Classification Inference	Run a live Rock Paper Scissors game with our pre-trained model on live stream or on event-based recordings.	classification_inference in Python classification_inference in C++

Note

Currently there is no sample dedicated to speed measurement. However, if you want to estimate the velocity of objects or an event stream for your application, you can have a look at the sample metavision_sparse_optical_flow (C++ version and Python version) or the sample metavision_spatter_tracking (C++ version and Python version).

Tools

Metavision SDK comes with a set of ready-to-use tools:

Name	Description
Metavision Studio	Advanced Graphical User Interface to visualize and record data streamed by Prophesee-compatible event-based vision systems
Metavision Viewer	Basic Graphical User Interface to visualize and record data streamed by Prophesee-compatible event-based vision systems
Metavision XYT	Displays events in a 3D space
Metavision Data Rate	Displays event rate from an event-based camera or from a recorded file
Camera Focusing	Focuses an event-based camera using a blinking pattern
Camera Calibration Pipeline	Estimates intrinsics and extrinsics parameters of EB camera using a blinking pattern
Ground Plane Calibration	Computes the 4x4 “world to Camera” transformation matrix
Metavision Platform Info	Prints information on the OS, connected devices and installed software
Metavision Software Info	Prints information on the installed software (version, date etc.)
Metavision File Info	Prints information about a RAW, DAT or HDF5 event file
File to CSV Converter	Converts a RAW, DAT or HDF5 event file to a CSV file
File to HDF5 Converter	Converts a RAW or DAT file to an HDF5 event file
File to video Converter	Converts a RAW, DAT or HDF5 event file to an AVI video
File to DAT Converter	Converts a RAW or HDF5 event file to a DAT-formatted file
Active Pixel Detection	Detects and masks active pixels for GenX320 sensor

Note

The source code of all those tools is available so that you can tune their behaviour according to your needs or use them as code examples to start programming. Hence those tools (except Studio) are also listed in our Code Samples section.