Machine Learning

Discover Metavision ML, a comprehensive machine learning framework for event-based vision.

Machine learning has revolutionized computer vision, allowing us to reach results that were impossible to imagine even a few years ago, especially in fields like object detection and classification. With Metavision ML, we can use the advantages of event-based vision together with the power of machine learning.

Currently available computer vision machine learning frameworks are heavily optimized to exploit the parallel operations of convolutional networks. However, due to the nature of event-based cameras, event-based data is typically temporally rich and spatially sparse, which makes it difficult to exploit it using the currently available machine learning frameworks. In Metavision ML module, we ensure that event-based data is correctly managed, and training loops and inference pipelines are designed to guarantee the compatibility between event-based data and generic machine learning frameworks.

Metavision ML uses PyTorch as a machine learning framework. This gives us the power of a well established framework, the convenience of Python and C++ integration, and the possibility to export your event-based model to other frameworks.

In Metavision ML, you will find all the tools required to ensure the best use of event-based data for machine learning:

  • tools to load event-based data in Python

  • tools to pre-process data to create tensors

  • training loops for object detection and classification, and optical flow

  • sample pipelines in C++ and Python for detection and tracking, including a pre-trained model for inference in automotive

  • sample pipeline in Python for optical flow, including a pre-trained model

  • conversion tools to create advanced and information-rich images from event-based data

  • large-scale datasets

  • tools for dataset creation: labeling, visualization, and evaluation