Note

This tutorial is showing how to get started with the SDK Python API. If you are interested in the C++ API, follow the tutorial Get Started using C++

Get Started using Python

In this Section, we will create a minimal sample to get started with Metavision SDK in Python. The goal of this sample is to introduce in the simplest possible way some basic concepts of Metavision SDK and create a first running example.

This tutorial will guide you in the creation of the source code step by step. For convenience, the full source code can also be found in <install-prefix>/share/metavision/sdk/core/python_samples/metavision_sdk_get_started when installing Metavision SDK from installer or packages. For other deployment methods, check the page Path of Samples.

Start the camera

The first operation we want to do is open the event-based camera or a pre-recorded file. In Metavision SDK, live cameras and pre-recorded RAW files are managed in the same way: by using the metavision_core.event_io.EventsIterator class. Let’s see how.

Create a python script, name it metavision_sdk_get_started.py and copy the following code in it:

from metavision_core.event_io import EventsIterator


def parse_args():
    import argparse
    """Parse command line arguments."""
    parser = argparse.ArgumentParser(description='Metavision SDK Get Started sample.',
                                     formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    parser.add_argument(
        '-i', '--input-raw-file', dest='input_path', default="",
        help="Path to input RAW file. If not specified, the live stream of the first available camera is used. "
        "If it's a camera serial number, it will try to open that camera instead.")
    args = parser.parse_args()
    return args


def main():
    """ Main """
    args = parse_args()

    # Events iterator on Camera or RAW file
    mv_iterator = EventsIterator(input_path=args.input_path, delta_t=1000)

    for evs in mv_iterator:
        print("Events are available!")


if __name__ == "__main__":
    main()

To execute it, you either need to plug in an event-based camera or use a pre-recorded RAW file.

Execute the following command if you want to process data from a live camera:

Linux

python3 metavision_sdk_get_started.py

Windows

python metavision_sdk_get_started.py

Execute the following command if you want to process data from a pre-recorded RAW file (from our Sample Recordings in this example):

Linux

python3 metavision_sdk_get_started.py -i monitoring_40_50hz.raw

Windows

python metavision_sdk_get_started.py -i monitoring_40_50hz.raw

Note

We suggest that you run the samples in this section with a pre-recorded RAW file, as some functions are less easy to demonstrate with a live camera.

If everything went fine, the output in your console should be similar to this:

...
Events are available!
Events are available!
Events are available!
Events are available!
Events are available!
Events are available!
Events are available!
...

Note

The large numbers of console outputs in this tutorial may slow down their execution. If this creates problems or prevents you from testing the code, we suggest you remove all print lines to improve execution speed.

Let’s analyze the sample line by line:

from metavision_core.event_io import EventsIterator

This line imports the metavision_core.event_io.EventsIterator class, which can be used to get the events from a pre-recorded RAW file but also from a live camera.

mv_iterator = EventsIterator(input_path=args.input_path, delta_t=1000)

Depending on the command line parameters, an instance of the metavision_core.event_io.EventsIterator class can be used to read from a file (args.input_path has been specified with a valid Path) or to stream from a camera (args.input_path contains a camera serial number or is empty). The source code will be the same whether the events are coming from a file or from a live camera.

This for loop simply continues printing Events are available while the iterator on the camera or file is providing event-buffers. For now, this EventsIterator runs until the recording is finished, in case of a pre-recorded RAW file, or forever in case of a live camera (at least, while the camera is plugged in).

for evs in mv_iterator:
    print("Events are available!")

To summarize, in this section, we learned how to open a camera or a file using the metavision_core.event_io.EventsIterator class, and how to start and stop a camera.

Get the events

In this section, we will learn how to get access to the events.

Copy the following code into your metavision_sdk_get_started.py file:

from metavision_core.event_io import EventsIterator


def parse_args():
    import argparse
    """Parse command line arguments."""
    parser = argparse.ArgumentParser(description='Metavision SDK Get Started sample.',
                                     formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    parser.add_argument(
        '-i', '--input-raw-file', dest='input_path', default="",
        help="Path to input RAW file. If not specified, the live stream of the first available camera is used. "
        "If it's a camera serial number, it will try to open that camera instead.")
    args = parser.parse_args()
    return args


def main():
    """ Main """
    args = parse_args()

    # Events iterator on Camera or RAW file
    mv_iterator = EventsIterator(input_path=args.input_path, delta_t=1000)

    global_counter = 0  # This will track how many events we processed
    global_max_t = 0  # This will track the highest timestamp we processed

    # Process events
    for evs in mv_iterator:
        print("----- New event buffer! -----")
        if evs.size == 0:
            print("The current event buffer is empty.")
        else:
            min_t = evs['t'][0]   # Get the timestamp of the first event of this callback
            max_t = evs['t'][-1]  # Get the timestamp of the last event of this callback
            global_max_t = max_t  # Events are ordered by timestamp, so the current last event has the highest timestamp

            counter = evs.size  # Local counter
            global_counter += counter  # Increase global counter

            print(f"There were {counter} events in this event buffer.")
            print(f"There were {global_counter} total events up to now.")
            print(f"The current event buffer included events from {min_t} to {max_t} microseconds.")
            print("----- End of the event buffer! -----")

    # Print the global statistics
    duration_seconds = global_max_t / 1.0e6
    print(f"There were {global_counter} events in total.")
    print(f"The total duration was {duration_seconds:.2f} seconds.")
    if duration_seconds >= 1:  # No need to print this statistics if the video was too short
        print(f"There were {global_counter / duration_seconds :.2f} events per second on average.")


if __name__ == "__main__":
    main()

Execute it. The output should be similar to this:

...
----- New event buffer! -----
There were 144241 events in this event buffer.
There were 19940983 total events up to now.
The current event buffer included events from 57000 to 57999 microseconds.
----- End of the event buffer! -----
----- New event buffer! -----
There were 111155 events in this event buffer.
There were 20052138 total events up to now.
The current event buffer included events from 58000 to 58999 microseconds.
----- End of the event buffer! -----
...

We can now compute some simple statistics on the events and print them:

for evs in mv_iterator:
    print("----- New event buffer! -----")
    if evs.size == 0:
        print("The current event buffer is empty.")
    else:
        min_t = evs['t'][0]   # Get the timestamp of the first event of this callback
        max_t = evs['t'][-1]  # Get the timestamp of the last event of this callback
        global_max_t = max_t  # Events are ordered by timestamp, so the current last event has the highest timestamp

        counter = evs.size  # Local counter
        global_counter += counter  # Increase global counter

        print(f"There were {counter} events in this event buffer.")
        print(f"There were {global_counter} total events up to now.")
        print(f"The current event buffer included events from {min_t} to {max_t} microseconds.")
        print("----- End of the event buffer! -----")

We have now access to the events produced by the metavision_core.event_io.EventsIterator class. With this, we will be able to write all our event-based algorithms.

Once our instance of metavision_core.event_io.EventsIterator has been stopped (that is, when the pre-recorded RAW file finishes), we can print global statistics of this file:

# Print the global statistics
duration_seconds = global_max_t / 1.0e6
print(f"There were {global_counter} events in total.")
print(f"The total duration was {duration_seconds:.2f} seconds.")
if duration_seconds >= 1:  # No need to print this statistics if the video was too short
    print(f"There were {global_counter / duration_seconds :.2f} events per second on average.")

To summarize, in this section, we learned how to get access to the events. We used it to count the number of events we receive and print their characteristics.

See also

If you want to know more about the EventsIterator class, check our SDK Core Tutorials In the Events Iterator tutorial, there is a section on how to create a HAL device to initialize an EventsIterator that will allow you to access facilities of your device (like biases, ROI etc.) to create more advanced applications.

Add a display

In the previous sections, we learned how to get access to the events. With this knowledge, it is now possible to create a display to visualize the output of the camera.

Doing it from scratch, while possible, would be complex and long. Luckily, the Metavision SDK contains a class that can help us with the visualization: metavision_sdk_core.PeriodicFrameGenerationAlgorithm.

Note

Event-based cameras do not produce frames, but a stream of independent events. To visualize these events, we must artificially build a frame by accumulating events over time. This can be done in different ways, but the easiest is to create a binary frame: we start with a frame where each pixel is set to zero, and we set to one the corresponding pixel every time we receive an event. We need to choose the frequency at which these frames are constructed, that is, the equivalent FPS, and how long we accumulate the events, that is, the accumulation time. See also the documentation on Event-Based Concepts for more information.

First, we need to include the metavision_sdk_core.PeriodicFrameGenerationAlgorithm class and the metavision_sdk_ui.Window window class.

from metavision_sdk_core import PeriodicFrameGenerationAlgorithm
from metavision_sdk_ui import EventLoop, BaseWindow, Window, UIAction, UIKeyEvent

Then we open a Graphical User Interface using a context manager:

mv_iterator = EventsIterator(input_path=args.input_path, delta_t=1000)
height, width = mv_iterator.get_size()  # Camera Geometry

# Window - Graphical User Interface
with Window(title="Metavision SDK Get Started", width=width, height=height, mode=BaseWindow.RenderMode.BGR) as window:
    def keyboard_cb(key, scancode, action, mods):
        if action != UIAction.RELEASE:
            return
        if key == UIKeyEvent.KEY_ESCAPE or key == UIKeyEvent.KEY_Q:
            window.set_close_flag()

    window.set_keyboard_callback(keyboard_cb)

Here we listen to the user for the q or Escape key, and use this command to exit the loop, effectively terminating the program. This finally allows us to stop our program even with a live camera, something that was not possible in the previous versions of our code.

We create the metavision_sdk_core.PeriodicFrameGenerationAlgorithm class by passing the resolution of the camera:

# Event Frame Generator
event_frame_gen = PeriodicFrameGenerationAlgorithm(sensor_width=width, sensor_height=height,
                                                   accumulation_time_us=accumulation_time_us)

def on_cd_frame_cb(ts, cd_frame):
    window.show(cd_frame)

event_frame_gen.set_output_callback(on_cd_frame_cb)

We then need to pass the events from the camera to the frame generator, so that it can accumulate them and create the frame for visualization. Every time some events are available, they will be passed to the frame generator.

# Process events
for evs in mv_iterator:
    # Dispatch system events to the window
    EventLoop.poll_and_dispatch()

    event_frame_gen.process_events(evs)

Execute this script. This is the expected result:

Note

When displaying a RAW file with the current version of this source code, the speed won’t be the real-time speed at which the file was recorded because EventsIterator is iterating over events as fast as possible. If you want to play a RAW file in a more realistic speed, you should use LiveReplayEventsIterator as demonstrated in the sample metavision_simple_viewer.

To summarize, in this section, we learned how to use the metavision_sdk_core.PeriodicFrameGenerationAlgorithm to create frames out of the events and display them.

Note

If you want to see how to apply some algorithms on the events before displaying them, check out the metavision_filtering sample that shows how PolarityFilterAlgorithm and RoiFilterAlgorithm can be applied on the stream of events.

Next steps

In this page, we introduced a minimal example of how to use the Metavision SDK to open a camera, compute some simple statistics on the received events, and visualize the events.

To build up on this code sample, you can explore more features of your camera, by accessing the Python binding of the facilities of your HAL device to tune the biases or enable an ESP feature.

We also encourage you to discover more complex examples in our samples page.