Installation of SDK Pro on Linux Ubuntu

This section describes how to install Metavision SDK Pro on Linux Ubuntu 20.04 and 22.04 64-bit. Compilation on other versions of Ubuntu or other Linux distributions was not tested. For those platforms some adjustments to this guide or to the code itself may be required (specially for non-Debian Linux).

Note

Metavision SDK Pro is our premium package giving access to the source code of all the SDK modules allowing you to customize the algorithms and compile on other platforms. Please visit our packaging page for more information and feel free to contact us for a quote if you are interested.

If you only need to have access to the source code of the Open modules (Base, Core, Core ML, Driver, UI, and HAL), compiling our open source project OpenEB will be enough. In that case, follow the specific guide on the Installation of OpenEB.

Supported Cameras

Metavision SDK Pro contains the source code of the Prophesee camera plugin, enabling to stream data from our event-based cameras and to read recordings of event-based data. The supported cameras are:

If you want to use a Prophesee EVK that is not in the list above, it might have been discontinued. Refer to the Prophesee EVKs Support section to find out which previous version of the SDK is supporting it.

If you own a third-party vendor event-based camera, refer to the Camera Plugin Installation page to see how it can be deployed and configured.

Required Configuration

Please note the base system requirements for using Metavision SDK:

  • Operating system: Linux Ubuntu 20.04 or 22.04 64-bit

  • Architecture: amd64 (a.k.a. x64)

  • Graphic card: must support OpenGL 3.0 minimum

  • CPU: must support AVX2

To install the SDK on Windows, check the page Installation of SDK Pro on Windows.

Required Artifacts

To install Metavision SDK Pro on Linux, you will need the source code archives Prophesee delivered to you:

  • OpenEB: metavision_open_x.y.z.tar

  • Advanced modules and standalone applications: metavision_sdk_advanced_sources_x.y.z.tar

Upgrading Metavision

If you are upgrading Metavision from a previous version, you should first read carefully the Release Notes as some changes may impact your usage of our SDK (e.g. API updates) and cameras (e.g. firmware update might be necessary).

Then, you need to clean your system from previously installed Prophesee software. If after a previous compilation, you chose to deploy the Metavision files in your system path, then go to the build folder in the source code directory and launch the following command to remove those files:

sudo make uninstall

In addition, remove Prophesee or Metavision folders and files in your system paths (/usr/include, /usr/lib, /usr/bin, /usr/share along with their /usr/local equivalents) and in your environment variables (PATH, PYTHONPATH and LD_LIBRARY_PATH).

Prerequisites

Installing General Dependencies

Install the following dependencies:

sudo apt update
sudo apt -y install apt-utils build-essential software-properties-common wget unzip curl git cmake
sudo apt -y install libopencv-dev libboost-all-dev libusb-1.0-0-dev libprotobuf-dev protobuf-compiler libeigen3-dev
sudo apt -y install libhdf5-dev hdf5-tools libglew-dev libglfw3-dev libcanberra-gtk-module ffmpeg

Optionally, if you want to run the tests, you need to install Google Gtest and Gmock packages. For more details, see Google Test User Guide :

sudo apt -y install libgtest-dev libgmock-dev

For the Python API, you will need Python and some additional libraries. If Python is not available on your system, install it (we support Python 3.8 and 3.9 on Ubuntu 20.04 and Python 3.9 and 3.10 on Ubuntu 22.04). Then install some extra libraries:

sudo apt -y install python3-pip python3-distutils
sudo apt -y install python3.X-dev  # where X is 8, 9 or 10 depending on your Python version (3.8, 3.9 or 3.10)
python3 -m pip install pip --upgrade
python3 -m pip install "opencv-python==4.5.5.64" "sk-video==1.1.10" "fire==0.4.0" "numpy==1.23.4" "h5py==3.7.0" pandas scipy
python3 -m pip install matplotlib "ipywidgets==7.6.5" pytest command_runner

Note

You can use anaconda to install Python and conda to manage your Python packages, but be sure to use a Python version that we support. You will need to adapt the library installation steps accordingly and use conda whenever we use pip.

In addition, check that conda PYTHONPATH contains /usr/lib/python3/dist-packages/. If not, add it. For example with sys.path.append() method of python module sys:

import sys
print(sys.path)
# Add /usr/lib/python3/dist-packages/ to PYTHONPATH if the output of print(sys.path) does not mention it.
sys.path.append("/usr/lib/python3/dist-packages/")

In this documentation, we chose to use pip as a package manager. We recommend using it with virtualenv to avoid conflicts with other installed Python packages.

The Python bindings of the C++ API rely on the pybind11 library (version >= 2.6.0).

Note

pybind11 is required only if you want to use the Python bindings of the C++ API. You can skip compiling these bindings by passing the argument -DCOMPILE_PYTHON3_BINDINGS=OFF during compilation (see step 3 of the compilation below). This allows you to skip installing pybind11, but you won’t be able to use our Python interface to the C++ API.

Unfortunately, there is no pre-compiled version of pybind11 available, so you need to install it manually:

wget https://github.com/pybind/pybind11/archive/v2.6.0.zip
unzip v2.6.0.zip
cd pybind11-2.6.0
mkdir build && cd build
cmake .. -DPYBIND11_TEST=OFF
cmake --build .
sudo cmake --build . --target install

Prerequisites for the CV module

To compile the XYT app from the CV sdk module. You’ll need to install Ogre3D, a graphic engine for 3D rendering.

  • From ubuntu 22.04:

sudo apt install libogre-1.12-dev libimgui-dev libfreetype-dev

  • On Ubuntu 20.04:

sudo add-apt-repository -y ppa:s-schmeisser/ogre-1.12
sudo apt install -y libogre-1.12
sudo apt install libogre-1.12-dev libimgui-dev libstb-dev libfreetype-dev

Prerequisites for the CV3D module

To use the CV3D module, you need to install a third-party Sophus library that is required to handle geometric transformation problems:

  • You need to install this archive version of Sophus:

    https://github.com/strasdat/Sophus/archive/1.22.10.zip

  • Unzip and go to the Sophus directory:

    unzip 1.22.10.zip
cd Sophus-1.22.10
mkdir build && cd build

  • Compile and install:

    cmake -DSOPHUS_USE_BASIC_LOGGING=ON ..
cmake --build . --config Release
sudo cmake --build . --target install

Prerequisites for the ML module

To use Machine Learning features, you need to install some additional dependencies. If you have some Nvidia hardware with GPUs, you can optionally install CUDA (11.6 or 11.7) and cuDNN to leverage them with PyTorch and LibTorch. Make sure that you install a version of CUDA that is compatible with your GPUs by checking Nvidia compatibility page.

Note

At the moment, we don’t support OpenCL and AMD GPUs.

Python Packages

You need to install PyTorch 1.13.1. Retrieve and execute the pip command of version 1.13.1 from the previous versions install guide section.

Then install some extra Python libraries:

python3 -m pip install "numba==0.56.3" "profilehooks==1.12.0" "pytorch_lightning==1.8.6" "tqdm==4.63.0" "kornia==0.6.8"
python3 -m pip install "llvmlite==0.39.1" "pycocotools==2.0.7" "seaborn==0.11.2" "torchmetrics==0.7.2" "pillow==9.3.0"

LibTorch for C++

To compile and run the neural network inference in a C++ pipeline, you need LibTorch (PyTorch’s C++ frontend)

  • Download the LibTorch version corresponding to your CUDA version (or take the CPU version if you don’t have CUDA):

  • Unzip the archive to a new folder LIBTORCH_DIR_PATH that you will reference when compiling the C++ inference sample. For example, for the CPU version:

    unzip libtorch-cxx11-abi-shared-with-deps-1.13.1+cpu.zip -d LIBTORCH_DIR_PATH

  • LibTorch is delivered with a copy of libgtest libraries. Those should be removed to avoid conflicts:

    rm LIBTORCH_DIR_PATH/libtorch/lib/libgtest.a
rm LIBTORCH_DIR_PATH/libtorch/lib/libgtest_main.a

Prerequisites for standalone apps

To be able to compile Metavision Studio, you need to install NodeJS. You can either follow the full NodeJS install guide of execute the following commands:

sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_16.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list
sudo apt-get update
sudo apt-get install nodejs -y

Warning

Metavision Studio is only compatible with amd64 (x64) architectures at the moment. If you try to compile/run it on other platforms (e.g. arm64) it will not work without modifications that we did not explore yet.

Preparation of the source code

First, extract the content of the archive metavision_open_x.y.z.tar:

tar -xvf metavision_open_x.y.z.tar

This will create the folder openeb-x.y.z containing OpenEB source code. The absolute path to this directory is called MV_SDK_SRC_DIR in the next sections.

Then extract the advanced source archive in MV_SDK_SRC_DIR:

tar -xvf metavision_sdk_advanced_sources_x.y.z.tar -C <MV_SDK_SRC_DIR>

Compilation

To compile SDK Pro, now follow those steps:

  1. open the MV_SDK_SRC_DIR directory

cd MV_SDK_SRC_DIR

  1. Create and open the build directory

mkdir build && cd build

  1. Generate the makefiles using CMake:

cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DUSE_SOPHUS=ON -DUSE_TORCH=ON -DTorch_DIR=<LIBTORCH_DIR_PATH>/share/cmake/Torch/

you can adapt this command depending on the module you choose to install:

  • to skip the compilation of the CV3D module, remove the Sophus option

  • to skip the compilation of the ML module, remove the Torch options

  • to skip the compilation of Metavision Studio, add -DCOMPILE_METAVISION_STUDIO=OFF

Note

If you want to specify to cmake which version of Python to consider, you should use the option -DPython3_EXECUTABLE=<path_to_python_to_use>. This is useful, for example, when you have a more recent version of Python than the ones we support installed on your system. In that case, cmake would select it and compilation might fail.

  1. Compile:

cmake --build . --config Release -- -j `nproc`

Configuration and deployment

Once the compilation is done, you have two options: you can choose to work directly from the build folder or you can deploy the SDK files in the system path (/usr/local/lib, /usr/local/include…).

  • Option 1 - working from build folder

    • To use the SDK directly from the build folder, you need to update some environment variables using this script (which you may add to your ~/.bashrc to make it permanent):

      source utils/scripts/setup_env.sh

    • Prophesee camera plugin is included in the SDK, but you still need to copy the udev rules files in the system path and reload them so that your camera is detected with this command:

    sudo cp <MV_SDK_SRC_DIR>/hal_psee_plugins/resources/rules/*.rules /etc/udev/rules.d
sudo udevadm control --reload-rules
sudo udevadm trigger

  • Option 2 - deploying in the system path

    • To deploy the SDK, launch the following command:

      sudo cmake --build . --target install

      Note

      You can also deploy the SDK files (applications, samples, libraries etc.) in a directory of your choice by using the CMAKE_INSTALL_PREFIX variable (-DCMAKE_INSTALL_PREFIX=<METAVISION_INSTALL_DIR>) when generating the makefiles in step 3 of the compilation. Similarly, you can configure the directory where the Python packages will be deployed using the PYTHON3_SITE_PACKAGES variable (-DPYTHON3_SITE_PACKAGES=<PYTHON3_PACKAGES_INSTALL_DIR>).

    • you also need to update LD_LIBRARY_PATH and HDF5_PLUGIN_PATH (which you may add to your ~/.bashrc to make it permanent):

      export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export HDF5_PLUGIN_PATH=$HDF5_PLUGIN_PATH:/usr/local/lib/hdf5/plugin  # On Ubuntu 20.04
export HDF5_PLUGIN_PATH=$HDF5_PLUGIN_PATH:/usr/local/hdf5/lib/plugin  # On Ubuntu 22.04

Note

If you are using a third-party camera, you need to install the plugin provided by the camera vendor and specify the location of the plugin using the MV_HAL_PLUGIN_PATH environment variable.

Running tests

Running the test suite is a sure-fire way to ensure you did everything well with your compilation and installation process.

The more modules you add the more tests will be run when you do this step.

  • Go to this page and click on the Download All button to retrieve an archive of all the files necessary to run the tests. Please be aware that the archive you’ll receive is approximately 5 Gb in size.

  • Extract and put the contents of this archive into <MV_SDK_SRC_DIR>. For instance, the correct path of sequence gen31_timer.raw should be <MV_SDK_SRC_DIR>/datasets/openeb/gen31_timer.raw.

  • Regenerate the makefiles with the test options enabled:

cd <MV_SDK_SRC_DIR>/build
cmake .. -DBUILD_TESTING=ON

  • Compile again:

cmake --build . --config Release -- -j `nproc`

  • Make sure you launched the utils/scripts/setup_env.sh mentioned before, and run the test suite:

ctest -C Release

Get started!

You are now ready to use Metavision SDK. The best way to start getting familiar with the event-based technology is to open an event-based camera with Metavision Studio to begin data collection and visualization. You can choose to dive directly in the SDK by following a getting started guide or looking at a Code Sample.