Machine Learning

Discover Metavision ML, a comprehensive machine learning framework for event-based vision.

Machine learning has revolutionized computer vision, allowing us to reach results that were impossible to imagine even a few years ago, especially in fields like object detection and classification. With Metavision ML, we can use the advantages of event-based vision together with the power of machine learning.

Currently available computer vision machine learning frameworks are heavily optimized to exploit the parallel operations of convolutional networks. However, due to the nature of event-based cameras, event-based data is typically temporally rich and spatially sparse, which makes it difficult to exploit it using the currently available machine learning frameworks. In Metavision ML module, we ensure that event-based data is correctly managed, and training loops and inference pipelines are designed to guarantee the compatibility between event-based data and generic machine learning frameworks.

Metavision ML uses PyTorch as a machine learning framework. This gives us the power of a well established framework, the convenience of Python and C++ integration, and the possibility to export your event-based model to other frameworks.

In Metavision ML, you will find all the tools required to ensure the best use of event-based data for machine learning:

• tools to load event-based data in Python

• tools to pre-process data to create tensors

• training loops for object detection and classification, and optical flow

• sample pipelines in C++ and Python for detection and tracking, including a pre-trained model for inference in automotive

• sample pipeline in Python for optical flow, including a pre-trained model

• conversion tools to create advanced and information-rich images from event-based data

• large-scale datasets

• tools for dataset creation: labeling, visualization, and evaluation

• Samples
  • Inference Pipeline of Detection and Tracking using Python
  • Inference Pipeline of Detection and Tracking using C++
  • Convert Bbox from a Sequential Text to NPY
  • Evaluation of Detection and Tracking Result with COCO KPIs
  • Export trained PyTorch classification model to TorchScript model
  • Export trained PyTorch detection model to TorchScript model
  • Export trained PyTorch flow model to TorchScript model
  • Inference Pipeline of Optical Flow
  • Inference Pipeline of Object Classification
  • Generate HDF5
  • Training of EB Classification Model
  • Training of Object Detection model
  • Training of Optical Flow model
  • Visualization of the Detection Results together with the Ground-truth
  • Visualization of Preprocessed HDF5 Tensor Files
  • Visualization of Moving MNIST Dataset
• Tutorials
  • Quick Start
  • Data processing
  • Inference
  • Training
  • Dataset creation
  • Using Python bindings
• Pre-trained Models
  • Event-to-Video Model
  • Corner Detection Model
  • Object Detection Models
  • Flow Models
  • Classification Models
• SDK ML Python API
  • SDK ML Classification API
  • SDK ML Detection And Tracking API
  • SDK ML Metrics API
  • SDK ML Python bindings API
  • SDK ML Core API
  • SDK ML Data API
  • SDK ML Detection API
  • SDK ML Flow API
  • SDK ML Preprocessing API
  • SDK ML Utils API
• SDK ML C++ API
  • SDK ML Algorithms
  • SDK ML Components
  • SDK ML Pipeline Builder
  • SDK ML Utils