Discover Metavision ML, a comprehensive machine learning framework for event-based vision.
Machine learning has revolutionized computer vision, allowing us to reach results that were impossible to imagine even a few years ago, especially in fields like object detection and classification. With Metavision ML, we can use the advantages of event-based vision together with the power of machine learning.
Currently available computer vision machine learning frameworks are heavily optimized to exploit the parallel operations of convolutional networks. However, due to the nature of event-based cameras, event-based data is typically temporally rich and spatially sparse, which makes it difficult to exploit it using the currently available machine learning frameworks. In Metavision ML module, we ensure that event-based data is correctly managed, and training loops and inference pipelines are designed to guarantee the compatibility between event-based data and generic machine learning frameworks.
Metavision ML uses PyTorch as a machine learning framework. This gives us the power of a well established framework, the convenience of Python and C++ integration, and the possibility to export your event-based model to other frameworks.
In Metavision ML, you will find all the tools required to ensure the best use of event-based data for machine learning:
tools to load event-based data in Python
tools to pre-process data to create tensors
training loops for object detection and classification, and optical flow
sample pipelines in C++ and Python for detection and tracking, including a pre-trained model for inference in automotive
sample pipeline in Python for optical flow, including a pre-trained model
conversion tools to create advanced and information-rich images from event-based data
tools for dataset creation: labeling, visualization, and evaluation
- Inference Pipeline of Detection and Tracking using Python
- Inference Pipeline of Detection and Tracking using C++
- Convert Bbox from a Sequential Text to NPY
- Evaluation of Detection and Tracking Result with COCO KPIs
- Export trained PyTorch classification model to TorchScript model
- Export trained PyTorch detection model to TorchScript model
- Export trained PyTorch flow model to TorchScript model
- Inference Pipeline of Optical Flow
- Inference Pipeline of Object Classification
- Generate HDF5
- Training of EB Classification Model
- Training of Object Detection model
- Training of Optical Flow model
- Visualization of the Detection Results together with the Ground-truth
- Visualization of Preprocessed HDF5 Tensor Files
- Visualization of Moving MNIST Dataset
- Pre-trained Models
- SDK ML Python API
- SDK ML C++ API