Installation of SDK from source on Linux Ubuntu

This section describes how to install Metavision SDK from source on Linux Ubuntu 18.04 and 20.04 64-bit. Compilation on other versions of Ubuntu or other Linux distributions was not tested. For those platforms some adjustments to this guide or to the code itself may be required (specially for non-Debian Linux).

Note

Contact us for a quote to acquire the source code of Metavision SDK. If you only need to have access to the source code of the Open modules (Base, Core, Core ML, Driver, UI, and HAL. See Metavision modules organization), compiling our open source project OpenEB will be enough. In that case, follow the specific guide on the Installation of OpenEB.

Requirements

Please note the base system requirements for using Metavision:

  • Operating system: Linux Ubuntu 18.04 or 20.04 64-bit

  • Architecture: amd64 (a.k.a. x64)

  • Graphic card: must support OpenGL 3.0 minimum

  • CPU: must support AVX2

For Windows, check the page Installation of SDK from source on Windows.

Prerequisites

Upgrading Metavision

First of all, read carefully the Release Notes as some changes may impact your usage of our SDK (e.g. API updates) and cameras (e.g. firmware update might be necessary).

Then, you need to clean your system from previously installed Prophesee software. If after a previous compilation, you chose to deploy the Metavision files in your system path, then go to the build folder in the source code directory and launch the following command to remove those files:

sudo make uninstall

In addition, make a global check in your system paths (/usr/lib, /usr/local/lib, /usr/include, /usr/local/include) and in your environment variables (PATH, PYTHONPATH and LD_LIBRARY_PATH) to remove occurrences of Prophesee or Metavision files.

Installing Dependencies

Install the following dependencies:

sudo apt update
sudo apt -y install apt-utils build-essential software-properties-common wget unzip curl git cmake
sudo apt -y install libopencv-dev libgtest-dev libboost-all-dev libusb-1.0-0-dev libeigen3-dev
sudo apt -y install libglew-dev libglfw3-dev libcanberra-gtk-module ffmpeg

If you want to run then tests, then you need to compile gtest package (this is optional):

cd /usr/src/gtest
sudo cmake .
sudo make
sudo make install

For the Python API, you will need Python and some additional libraries. If Python is not available on your system, install it (we support Python 3.6 and 3.7 on Ubuntu 18.04 and Python 3.7 and 3.8 on Ubuntu 20.04). Then install some extra libraries:

sudo apt -y install python3-pip python3-distutils
python3 -m pip install pip --upgrade
python3 -m pip install "opencv-python>=4.5.5.64" "sk-video==1.1.10" "fire==0.4.0" "numpy<=1.21" pandas scipy h5py pytest
python3 -m pip install jupyter jupyterlab matplotlib "ipywidgets==7.6.5"

Note

You can use anaconda to install Python and conda to manage your Python packages, but be sure to use a Python version that we support. You will need to adapt the library installation steps accordingly and use conda whenever we use pip. In this documentation, we chose to use pip as a package manager. We recommend using it with virtualenv to avoid conflicts with other installed Python packages.

The Python bindings rely on the pybind11 library (version >= 2.6.0).

Note

pybind11 is required only if you want to use the Python bindings of our C++ API. You can skip compiling these bindings by passing the argument -DCOMPILE_PYTHON3_BINDINGS=OFF during compilation (see step 3 below). This allows you to skip installing pybind11, but you won’t be able to use our Python interface.

Unfortunately, there is no pre-compiled version of pybind11 available, so you need to install it manually:

wget https://github.com/pybind/pybind11/archive/v2.6.0.zip
unzip v2.6.0.zip
cd pybind11-2.6.0
mkdir build && cd build
cmake .. -DPYBIND11_TEST=OFF
cmake --build .
sudo cmake --build . --target install

Prerequisites for the CV3D module

If you acquired the CV3D module, you need to install a third-party Sophus library that is required to handle geometric transformation problems:

  • Clone the Sophus repository:

    git clone https://github.com/strasdat/Sophus.git
    
  • Go to the Sophus directory:

    cd Sophus
    mkdir build && cd build
    
  • Compile and install:

    cmake ..
    cmake --build . --config Release
    sudo cmake --build . --target install
    

Prerequisites for the ML module

To use Machine Learning features, you need to install some additional dependencies. If you have some Nvidia hardware with GPUs, install CUDA (10.2 or 11.1) and cuDNN to leverage them with PyTorch and LibTorch. Make sure that you install a version of CUDA that is compatible with your GPUs by checking Nvidia compatibility page.

Note

At the moment, we don’t support OpenCL and AMD GPUs.

Packages for Python

Metavision ML requires PyTorch 1.8.2 LTS. This version was deprecated by PyTorch team but can still be downloaded in the Previous Versions page of pytorch.org (in future releases of Metavision ML, more recent version of PyTorch will be leveraged). Retrieve and execute the pip command for the installation:

PyTorch install

Metavision ML (ML and Core ML modules) has some extra Python requirements that can be installed with pip:

python3 -m pip install numba llvmlite profilehooks "pytorch_lightning==1.5.10"
python3 -m pip install "pycocotools==2.0.4" "tqdm==4.63.0"
python3 -m pip install "torchmetrics==0.7.2" "seaborn==0.11.2" "kornia==0.6.1"

LibTorch for C++

To compile and run the neural network inference in a C++ pipeline, you need LibTorch (PyTorch’s C++ frontend)

  • Download the LibTorch version corresponding to your CUDA version (or take the CPU version if you don’t have CUDA):

  • Unzip the archive to a new folder LIBTORCH_DIR_PATH that you will reference when compiling the C++ inference sample. For example, for the CPU version:

    unzip libtorch-cxx11-abi-shared-with-deps-1.10.0+cpu.zip -d LIBTORCH_DIR_PATH
    
  • LibTorch is delivered with a copy of libgtest libraries. Those should be removed to avoid conflicts:

    rm LIBTORCH_DIR_PATH/libtorch/lib/libgtest.a
    rm LIBTORCH_DIR_PATH/libtorch/lib/libgtest_main.a
    

Compilation

Compilation of OpenEB

The first step to compile the whole Metavision SDK from source is to first compile OpenEB. You should use the OpenEB archive (openeb_x.y.z.tar) that was included in your delivery.

  1. Extract the source code of OpenEB in an openeb folder (absolute path to this directory is called OPENEB_SRC_DIR in next sections):

tar xvf openeb_x.y.z.tar
  1. Create and open the build directory

cd openeb
mkdir build && cd build
  1. Generate the makefiles using CMake:

cmake .. -DBUILD_TESTING=OFF
  1. Compile:

cmake --build . --config Release -- -j 4

Compilation of the standalone apps

Next, compile the standalone applications (today, Metavision Studio). Extract the archive (metavision_sdk_sources_standalone_apps_x.y.z.tar) that was included in your source delivery in OPENEB_SRC_DIR.

Note

Standalone apps source delivery is restricted to customers who acquired our SDK Source Code offer. Contact us for a quote.

Before compilation, you need to install the packages necessary to compile Metavision Studio:

curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
sudo apt update && sudo apt install nodejs
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt update && sudo apt install yarn
  • Generate the makefiles for the new code with CMake:

    cmake .. -DBUILD_TESTING=OFF
    
  • Compile:

    cmake --build . --config Release -- -j 4
    

Compilation of the advanced modules

  • Extract the archives of the advanced modules that were included in your source delivery in OPENEB_SRC_DIR folder:

    tar -C <OPENEB_SRC_DIR> -xvf metavision_sdk_sources_xxx.tar
    tar -C <OPENEB_SRC_DIR> -xvf metavision_sdk_sources_yyy.tar
    ...
    

    Note

    Advanced Modules source delivery is restricted to customers who acquired our SDK Source Code offer. Contact us for a quote.

  • Go to the build directory:

    cd <OPENEB_SRC_DIR>/build
    
  • Run CMake to detect the advanced modules you extracted:

    cmake .. -DBUILD_TESTING=OFF [ADDITIONAL_OPTIONS]
    

    With the value of ADDITIONAL_OPTIONS that depends on the modules you are compiling:

    • if you are compiling the CV3D module, you need to add the option -DUSE_SOPHUS=ON

    • if you are compiling the ML module, you need to add the option -DUSE_TORCH=ON -DTorch_DIR=<LIBTORCH_DIR_PATH>/share/cmake/Torch/

  • Compile:

    cmake --build . --config Release -- -j 4
    

Configuration and deployment

  • To use Metavision Intelligence directly from the build folder, you need to update some environment variables using this script (which you may add to your ~/.bashrc to make it permanent):

    source utils/scripts/setup_env.sh
    

    Optionally, you can deploy the OpenEB files in the system path (/usr/local/lib, /usr/local/include…) to use them as 3rd party dependency in some other code with the following command:

    sudo cmake --build . --target install
    

    In that case, you will also need to update LD_LIBRARY_PATH:

    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
    

    And if you want to update this path permanently, you should add the previous command in your ~/.bashrc

    Note

    You can also deploy the OpenEB files (applications, samples, libraries etc.) in a directory of your choice by using the CMAKE_INSTALL_PREFIX variable (-DCMAKE_INSTALL_PREFIX=<OPENEB_INSTALL_DIR>) when generating the makefile. Similarly, you can configure the directory where the Python packages will be deployed using the PYTHON3_SITE_PACKAGES variable (-DPYTHON3_SITE_PACKAGES=<PYTHON3_PACKAGES_INSTALL_DIR>).

  • Since OpenEB 3.0.0, Prophesee camera plugins are included in OpenEB. In the previous step, if you did not perform the deployment step (sudo cmake --build . --target install) and instead used “setup_env.sh”, then you need to copy the udev rules files used by Prophesee cameras in the system path and reload them so that your camera is detected with this commands:

    sudo cp $METAVISION_SRC_DIR/hal_psee_plugins/resources/rules/*.rules /etc/udev/rules.d
    udevadm control --reload-rules
    udevadm trigger
    

    If you are using a third-party camera, you need to install the plugin provided by the camera vendor and specify the location of the plugin using the MV_HAL_PLUGIN_PATH environment variable.

Running tests

Running the test suite is a sure-fire way to ensure you did everything well with your compilation and installation process.

The more modules you add the more tests will be run when you do this step.

  • Go to this page to download the files necessary to run the tests. Click the Download button on the top right of the page. The obtained archive weighs around 3 Gb.

  • Extract and put the contents of this archive into <OPENEB_SRC_DIR>/. For instance, the correct path of sequence gen31_timer.raw should be <OPENEB_SRC_DIR>/datasets/openeb/gen31_timer.raw.

  • Regenerate the makefiles with the test options on:

cd <OPENEB_SRC_DIR>/build
cmake .. -DBUILD_TESTING=ON
  • Compile again:

cmake --build . --config Release -- -j 4
  • Make sure you launched the utils/scripts/setup_env.sh mentioned before, and run the test suite:

ctest -C Release

Get started!

You are now ready to use Metavision Intelligence. The best way to start getting familiar with the event-based technology is to open an event-based camera with Metavision Studio to begin data collection and visualization. You can choose to dive directly in the SDK by following a Tutorial or looking at a Code Sample.