Metavision Studio

Metavision Studio is the perfect tool to start with, whether you have an event-based camera or not.

It features a Graphical User Interface allowing to visualize and record data streamed by Prophesee-compatible event-based vision systems. You can try it out by playing one of the event files (RAW or HDF5) provided in our sample recordings page. If you own one of our Evaluation Kits or a camera from our partner, you can visualize the events, adjust the display parameters and tune multiple camera settings.

Note

Metavision Studio is not part of OpenEB, but the more basic Metavision Viewer application can be used instead.

Main features

Stream and display events from a live camera
Visualize events from a recording in the frame rate of your choice (normal, slow-motion, high-speed…)
Configure display parameters (accumulation time, color theme)
Control sensor pixels settings (biases)
Set a Region Of Interest (ROI)
Configure Event Signal Processing (ESP) (Anti-Flicker, Event Trail Trail Filter and Event Rate Controller)
Configure Trigger interfaces (Trigger In and Trigger Out)
Record data from a live camera to RAW file
Cut recording to keep only relevant data
Export events recording to HDF5 file or to AVI video
Serialize/deserialize camera configuration (save/load camera settings)

Starting Studio

Metavision Studio can be started by typing metavision_studio in the command line prompt of your operating system. On Windows, you can also launch it from the Windows Menu or the Search Bar.

On your first connexion, you will be offered to follow a guided tour to discover how to use Studio:

This guide gives you information to start reading and recording event-based data as well as configuring your camera settings. Note that you will be able to access it anytime by selecting “Help > Getting Started Guide” menu item.

First visualization of a recording

To start with Studio, download an event file from our Sample Recordings (RAW or HDF5). For example, choose laser.raw which is a laser recorded using an Evk3 Gen4.1 (HD sensor). When you open the file in Studio, it will automatically be played. The default color theme is a black background with ON and OFF events displayed respectively as white and blue pixels.

Now, open the Settings panel on the right and look at the Information, Statistics and Display sections. In the Information section, you can see that it was recorded with a Gen4.1 sensor of HD resolution (1280x720). In the Statistics section, you can see a live estimation of the data rate which is quite constant for this recording (around 38 Mev/s). Finally, in the Display section, you can change the way the frames are built and displayed from the events. For example, change Accumulation Time to 1ms and Frame Rate to 1000 (0.03x) to see the recording in slow motion:

Warning

When opening the recording with Studio on Windows, you may get a not an existing file error:

------------------------------------------------
Metavision SDK Driver exception

Error 103001: Opening file at c:\Users\name\Recordings\??.raw: not an existing file.
No such file or directory.
------------------------------------------------

This error is probably caused by some non-ANSI characters in the file path (either in the folder or the file name) like characters with accents or asian characters (chinese, japanese etc.). Until the SDK is enhanced to support those file paths properly, the workaround is to store your recording in pure-ANSI paths.

Note

To understand the concept of accumulation time and more generally to get more familiar with our way of managing events, go through the event-based concepts page where we explain how frames are generated from events.

First visualization from live camera

If you own an event-based camera, plug it to an USB3 port and open the camera in Studio. If no camera is detected and you see the message “No cameras available, please check that a camera is connected”, please refer to our camera troubleshooting FAQ entry.

You should now see live events streamed from your camera. You can adjust the color theme to your liking in the Display section of the settings. In the image below, we chose the Light theme: ON and OFF events are displayed respectively as blue and black pixels on a white background. If you are pointing your camera to a static scene, you won’t see anything besides some background noise. To get some relevant events, you can point the camera to yourself and wave your hand:

Metavision Studio streaming from live camera

Note

In rare cases, your EVK device may show up as a USB2 device or present the following error: EVKx device is not enumerated as a USB 3 SuperSpeed device. To resolve this issue, please disconnect the cable from the PC USB port and re-insert the cable with increased speed. For more information, please refer to our FAQ.

The image shown above is quite sharp and does not show much background noise. To reach such a result, focus your camera by adjusting the aperture and focus distance of your objective (availability of those settings depends on your objective). To reduce the background noise, you can open the biases settings and adjust bias_fo while following the event rate and the display to see how the noise is impacted. To get more information about the biases, please refer to the sensor biases page.

After focusing and bias adjustment, Studio might still show an image with many unexpected events like in this example:

This could be caused by your lighting conditions. Some artificial lights are flickering and creating many events on the sensors. In the image above, you can even see some horizontal artefacts caused by a saturation of the sensor or by the Event Rate Controller (ERC) of the sensor (on Gen4.0 and newer) which might be configured to limit the event rate. In such a situation, change your lighting device if you have the possibility to do so. The best non-flickering light source is any halogen lighting, but if you want a LED source, you must double check it is not flickering (lots of them are using PWM modulation for dimming, and produce flicker). If you can not choose your lighting device, then on Gen4.1 and IMX636 sensors, you can enable the Anti-Flicker sensor filter available in the Settings panel of Studio. To get more information on flicker mitigation, check the Application Note in the Knowledge Center.

Warning

Like explained in the ERC section, this filter degrades the quality of the signal (this can be seen visually in the screenshot above). So if you don’t enable the ERC, you will get all the events produced by the camera, without any filter. Beware that in that case, you might reach the bandwidth and computation limits of the of the devices (camera and laptop) and their connexions (internal to the camera and USB between camera and laptop). In that situation, the display will start lagging and it may even lead to data loss or corruption. So before pushing the event rate to high values, make sure that you are gathering relevant events (see next section on recording).

First recording from live camera

Now that you successfully streamed from an event-based camera, you can do your first recordings. Note that some checks should be performed to get good quality data.

First, review your general setup:

Camera installation: when possible, mount your camera on a tripod stand to avoid any spurious motion during acquisition
Lighting conditions: like mentioned in the previous section, make sure you don’t have unexpected events due to flickering light source.
Focus adjustment: to help you focusing your camera, you can use the metavision_blinking_pattern_focus application

Then adjust some sensor settings to enhance the quality of your data:

Sensor pixels settings (biases): depending on your applications requirements and conditions (higher speed, lower background activity, higher contrast sensitivity threshold, etc.), you should adjust the camera biases.
Region Of Interest: whenever possible you should limit the area of the sensor to the pixels that might gather relevant events for your application. For example, if you are tracking vehicle on a road and you are unable to adjust your objective lens to see only the road, you can configure a ROI in Metavision Studio to exclude non-relevant areas (sidewalks, sky etc.).
Events filtering: if your camera is using a Gen4, IMX636 or GenX320 sensor, leverage the Event Signal Processing (ESP) block that provide some event filtering features: Anti-Flicker, Event Trail Filter and Event Rate Controller (ERC).

When you are satisfied with your setup, start doing some recordings with Studio. You will get some RAW files that you can then play-back in Studio with different frame rate and accumulation time.

Warning

Keep in mind that the RAW files you will record don’t depend on the accumulation time you configured in Studio. It is important to understand that this accumulation time is only a visualization option similar to watching a video file in slow motion.

Out of curiosity, you can take a look at the actual events contained in those RAW files. To do so, use the metavision_file_to_csv sample to generate a CSV file and check its content to see the x,y,p,t tuples:

$ more my_first_recording.csv
382,341,0,5012
548,716,0,5031
990,625,0,5042
1162,122,0,5043
524,665,0,5043
987,606,1,5049
209,73,0,5052
878,504,0,5062
(...)

In this example, we see that the very first event recorded was an OFF event located at coordinates (x=382,y=341) and timestamped at t=5012us.

Saving and loading camera settings

Once you have configured some camera settings, you might want to save those settings for another session in Studio or to use them in another application of the SDK. For example, in the following screenshot, we can see that bias_hpf was set to 30 and that an ROI was configured. You can now save the camera settings by clicking on the icon “Save Settings” on the bottom of the display or choose the “Save camera settings” in the File menu:

If you look at the content of the saved file, you will find the biases, the ROI and all the other settings that were applied to the camera. Note that some of them are not yet available in Metavision Studio like Digital Event Mask, Digital Crop (see C++ API: I_DigitalCrop) and the camera synchronization mode. Note also that depending on the way the Studio GUI was built, some settings won’t refresh the settings panel (e.g. loading an ROI won’t fill-in the values of the current ROI in the settings).

Once you have a JSON settings files that suits you, you can load it in Studio whenever you need, but you can also use it in Metavision Viewer and with most of our C++ samples.

Next steps

To go further, you can now explore the various use cases and applications that you may consider focusing on, alongside the other tools available.

If you can’t wait to start coding, check our getting started, programming guides and code samples and build your own solution using our C++ API or Python API.

Warning

Metavision Studio is leveraging Electron framework that is using multiple 3rd party dependencies. A list of these Licenses is included in the file SDK_DEPENDENCIES located in the folder share/metavision/licensing within your installation path and also in this dedicated page of our documentation site.