SDK Core Utils

class Metavision::CDFrameGenerator

Utility class to display CD events that handles multithreading and is built on the top of PeriodicFrameGenerator.

Public Functions

CDFrameGenerator(long width, long height, bool process_all_frames = false)

Default constructor.

Parameters

  • width: Width of the image (in pixels)

  • height: Height of the image (in pixels)

  • process_all_frames: If true, it will process all frames, not just the latest one. Note that if true it can slow down the process.

~CDFrameGenerator()

Destructor.

void set_colors(const cv::Scalar &background_color, const cv::Scalar &on_color, const cv::Scalar &off_color, bool colored = false)

Sets the color used to generate the frame.

By default, the frame generated will be 8 bits single channel with the default colors grey, white, black as background, on and off colors respectively. If the parameter colored is false, then the generated frame will be 8 bits single channel and the first channel will be used to define each colors (i.e. the other channels will be ignored). If the parameter colored is true, then the generated frame will be 8 bits three channels.

Parameters

  • background_color: Color used as background, when no events were received for a pixel, default: grey cv::Scalar::all(128)

  • on_color: Color used for on events, default: white cv::Scalar::all(255)

  • off_color: Color used for off events, default: black cv:Scalar::all(0)

  • colored: If the generated frame should be single or three channels

void set_color_palette(const Metavision::ColorPalette &palette)

Sets the color used to generate the frame.

Parameters

  • palette: The Prophesee’s color palette to use

void add_events(const Metavision::EventCD *begin, const Metavision::EventCD *end)

Adds the buffer of events to be displayed.

Parameters

  • begin: Beginning of the buffer of events

  • end: End of the buffer of events

void set_display_accumulation_time_us(timestamp display_accumulation_time_us)

Sets the time interval to display events.

The events shown at each refresh are such that their timestamps are in the last ‘display_accumulation_time_us’ microseconds from the last received event timestamp.

Parameters

  • display_accumulation_time_us: The time interval to display events from up to now (in us).

bool start(std::uint16_t fps, const PeriodicFrameGenerationAlgorithm::OutputCb &cb)

Starts the generator thread.

Return

true if the thread started successfully, false otherwise. Also returns false, if the thread is already started.

Parameters

  • fps: Frame rate

  • cb: Function to call every time a new frame is available. It takes in input the time (in us) of the new frame and the frame. The frame passed as a parameter is guaranteed to be available and left untouched until the next time the callback is called. This means that you don’t need to make a deep copy of it, if you only intend to use the frame until the next one is made available by the callback.

bool stop()

Stops the generator thread.

Return

true if the thread has been stopped successfully, false otherwise. Return false if the thread had not been previously started

void reset()

Resets the frame generator state.

struct Metavision::RGBColor

Struct that represents a color in RGB colorspace.

Public Members

double r

a fraction between 0 and 1

double g

a fraction between 0 and 1

double b

a fraction between 0 and 1

struct Metavision::HSVColor

Struct that represents a color in HSV colorspace.

Public Members

double h

angle in degrees

double s

a fraction between 0 and 1

double v

a fraction between 0 and 1

RGBColor Metavision::hsv2rgb(HSVColor hsv)

Utility function to convert from HSV to RGB colorspace.

Return

RGBColor the color converted in RGB colorspace

Parameters

  • hsv: A color in HSV colorspace

HSVColor Metavision::rgb2hsv(RGBColor rgb)

Utility function to convert from RGB to HSV colorspace.

Return

HSVColor the color converted in HSV colorspace

Parameters

  • rgb: A color in RBB colorspace

enum Metavision::ColorPalette

Enum class representing available color palettes.

Values:

enumerator Light
enumerator Dark
enumerator Gray
enum Metavision::ColorType

Enum class representing one of the possible type of colors.

Values:

enumerator Background
enumerator Positive
enumerator Negative
enumerator Auxiliary
const RGBColor &Metavision::getColor(const ColorPalette &palette, const ColorType &type)

Gets a color given a palette and the color type.

Return

The color associated to the given palette and type

Parameters

  • palette: The requested color palette

  • type: The requested color type

const RGBColor &Metavision::getColor(const ColorPalette &palette, const std::string &name)

Gets a color given a palette and the color name.

Return

The color associated to the given palette and name

Parameters

  • palette: The requested color palette

  • name: The requested color name

class Metavision::CvColorMap

Wrapper around cv::applyColorMap to avoid reallocating memory for the colormap at each call.

The colormap look-up-table is created only once, when the class is constructed. Then the parenthesis operator allows to apply it on an input image.

See implementation of “void ColorMap::operator()(InputArray _src, OutputArray _dst) const” in https://github.com/opencv/opencv/blob/master/modules/imgproc/src/colormap.cpp

Public Functions

CvColorMap(unsigned int cmap = 2)

Initializes the colormap.

Parameters

  • cmap: Colormap to apply. See cv::ColormapTypes. It’s cv::COLORMAP_JET by default

void operator()(const cv::Mat &src, cv::Mat &dst) const

Applies the colormap on a given image.

Parameters

  • src: Source image, grayscale or colored of type CV_8UC1 or CV_8UC3

  • dst: Result is the colormapped source image. Note: cv::Mat::create is called on dst

class Metavision::CvVideoRecorder

A simple threaded video recorder using OpenCV routines.

Public Functions

~CvVideoRecorder()

Records all remaining frames then destroys the object.

bool start()

Starts the recording thread.

void stop()

Stops adding data to the recording queue through the write methods.

The recorder thread remains active until all data added in the queue have been dumped.

void write(cv::Mat &data)

Pushes the input frame for writing.

This method does nothing if the recorder thread is not active

bool is_recording()

Returns if the recording thread is ongoing.

class Metavision::VideoWriter : public VideoWriter

Video writer class.

The class provides C++ API for writing video files or image sequences.

Note

This class fixes a bug in the OpenCV MJPEG encoder and it is recommended to use it over the original cv::VideoWriter.

Public Functions

VideoWriter()

Default constructors.

VideoWriter(const cv::String &filename, int fourcc, double fps, cv::Size frameSize, bool isColor = true)

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts. Tips:

  • With some backends fourcc=-1 pops up the codec selection dialog from the system.

  • To save image sequence use a proper filename (eg. img_%02d.jpg) and fourcc=0 OR fps=0. Use uncompressed image format (eg. img_%02d.BMP) to save raw frames.

  • Most codecs are lossy. If you want lossless video file you need to use a lossless codecs (eg. FFMPEG FFV1, Huffman HFYU, Lagarith LAGS, etc…)

  • If FFMPEG is enabled, using codec=0; fps=0; you can create an uncompressed (raw) video file.

Parameters

  • filename: Name of the output video file.

  • fourcc: 4-character code of codec used to compress the frames. For example, VideoWriter::fourcc(‘P’,’I’,’M’,’1’) is a MPEG-1 codec, VideoWriter::fourcc(‘M’,’J’,’P’,’G’) is a motion-jpeg codec etc. List of codes can be obtained at Video Codecs by FOURCC page. FFMPEG backend with MP4 container natively uses other values as fourcc code: see ObjectType, so you may receive a warning message from OpenCV about fourcc code conversion.

  • fps: Framerate of the created video stream.

  • frameSize: Size of the video frames.

  • isColor: If it is not zero, the encoder will expect and encode color frames, otherwise it will work with grayscale frames.

VideoWriter(const cv::String &filename, int apiPreference, int fourcc, double fps, cv::Size frameSize, bool isColor = true)

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts. The apiPreference parameter allows to specify API backends to use. Can be used to enforce a specific reader implementation if multiple are available: e.g. cv::CAP_FFMPEG or cv::CAP_GSTREAMER.

VideoWriter(const cv::String &filename, int fourcc, double fps, const cv::Size &frameSize, const std::vector<int> &params)

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts. The params parameter allows to specify extra encoder parameters encoded as pairs (paramId_1, paramValue_1, paramId_2, paramValue_2, … .) see cv::VideoWriterProperties

VideoWriter(const cv::String &filename, int apiPreference, int fourcc, double fps, const cv::Size &frameSize, const std::vector<int> &params)

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

~VideoWriter()

Default destructor.

The method first calls VideoWriter::release to close the already opened file.

bool open(const cv::String &filename, int fourcc, double fps, cv::Size frameSize, bool isColor = true)

Initializes or reinitializes video writer.

The method opens video writer. Parameters are the same as in the constructor VideoWriter::VideoWriter.

The method first calls VideoWriter::release to close the already opened file.

Return

true if video writer has been successfully initialized

bool open(const cv::String &filename, int apiPreference, int fourcc, double fps, cv::Size frameSize, bool isColor = true)

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

bool open(const cv::String &filename, int fourcc, double fps, const cv::Size &frameSize, const std::vector<int> &params)

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

bool open(const cv::String &filename, int apiPreference, int fourcc, double fps, const cv::Size &frameSize, const std::vector<int> &params)

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

bool isOpened() const

Returns true if video writer has been successfully initialized.

void release()

Closes the video writer.

The method is automatically called by subsequent VideoWriter::open and by the VideoWriter destructor.

VideoWriter &operator<<(const cv::Mat &image)

Stream operator to write the next video frame.

See

write

VideoWriter &operator<<(const cv::UMat &image)

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

See

write

void write(cv::InputArray image)

Writes the next video frame.

The function/method writes the specified image to video file. It must have the same size as has been specified when opening the video writer.

Parameters

  • image: The written frame. In general, color images are expected in BGR format.

bool set(int propId, double value)

Sets a property in the VideoWriter.

Return

true if the property is supported by the backend used by the VideoWriter instance.

Parameters

  • propId: Property identifier from cv::VideoWriterProperties (eg. cv::VIDEOWRITER_PROP_QUALITY) or one of videoio_flags_others

  • value: Value of the property.

double get(int propId) const

Returns the specified VideoWriter property.

Return

Value for the specified property. Value 0 is returned when querying a property that is not supported by the backend used by the VideoWriter instance.

Parameters

  • propId: Property identifier from cv::VideoWriterProperties (eg. cv::VIDEOWRITER_PROP_QUALITY) or one of videoio_flags_others

cv::String getBackendName() const

Returns used backend API name.

Note

Stream should be opened.

class Metavision::FrameComposer

This class implements an object capable of displaying a big image containing many images in an openGL window.

You need to first call method FrameComposer::add_new_subimage_parameters for each small image that you want to display in the big one. It will create an ImageParams object that will store the reference to the small image but also its position and other fields. Every time you call FrameComposer::update_subimage, FrameComposer will gather all ImageParams, put image at the right place in the big one and copy that in the output image.

Ideally the class takes integer pixels as inputs (CV_8UC1 or CV_8UC3). In case of floating point values, the cv::Mat is converted to an int type, which might slightly slow down the process.

Public Functions

FrameComposer(const cv::Vec3b &background_color = cv::Vec3b(0, 0, 0))

Constructor.

Parameters

  • background_color: Default color of the composition background (CV_8UC3)

~FrameComposer()

Destructor.

unsigned int add_new_subimage_parameters(int top_left_x, int top_left_y, const ResizingOptions &resize_options, const GrayToColorOptions &gray_to_color_options)

Selects a sub-part of the final composed image and defines its preprocessing options.

Warning

It’s possible to define sub-images with overlapping regions of interests. Each update of a sub-image overwrites the corresponding subpart of the image, and thus also overwrites the overlapping areas. Thus, if a small sub-image is located on top of a larger sub-image, it will only be visible if its resfresh rate is larger than or equal to the one of the larger sub-image, and ist update with update_subimage is called at the end

Return

Reference (integer) of the sub-image to update later with update_subimage

Parameters

  • top_left_x: X-position of the top-left corner of the image in the composition

  • top_left_y: Y-position of the top-left corner of the image in the composition

  • resize_options: Options used to resize the image

  • gray_to_color_options: Options used to rescale and/or apply a colormap on the grayscale image

bool update_subimage(unsigned int img_ref, const cv::Mat &image)

Update the sub-part of the final image corresponding to the reference img_ref.

Parameters

  • img_ref: Reference ID of the sub-image, defined in the function add_new_subimage_parameters

  • image: Image to display at the corresponding location in the full composed image If the type is different than CV_8UC3 or CV_8UC1, then a conversion is performed

const cv::Mat &get_full_image() const

Gets the full image.

Return

The composed image

const int get_total_width() const

Gets width of the output image.

Return

The width of the output image

const int get_total_height() const

Gets height of the output image.

Return

The height of the output image

struct GrayToColorOptions

Settings used to rescale and/or apply a colormap on a grayscale image.

Public Members

unsigned char min_rescaling_value = 0

Min grayscale value for the rescaling.

unsigned char max_rescaling_value = 255

Max grayscale value for the rescaling.

int color_map_id = -1

Colormap to apply:

  • None = -1

  • COLORMAP_AUTUMN = 0

  • COLORMAP_BONE = 1

  • COLORMAP_JET = 2

  • COLORMAP_WINTER = 3

  • COLORMAP_RAINBOW = 4

  • COLORMAP_OCEAN = 5

  • COLORMAP_SUMMER = 6

  • COLORMAP_SPRING = 7

  • COLORMAP_COOL = 8

  • COLORMAP_HSV = 9

  • COLORMAP_PINK = 10

  • COLORMAP_HOT = 11

struct ResizingOptions

Settings used to resize or crop an image.

Public Members

const unsigned int height

New size of the image in the final composition.

const bool enable_crop

Whether to enable cropping the image to the specified width and height (maintains the center)

const InterpolationType interp_type

Type of interpolation used to resize images.

template<typename timestamp_type>
class Metavision::MostRecentTimestampBufferT

Class representing a buffer of the most recent timestamps observed at each pixel of the camera.

A most recent timestamp buffer is also called time surface.

Note

The interface follows the one of cv::Mat

Public Functions

MostRecentTimestampBufferT()

Default constructor.

MostRecentTimestampBufferT(int rows, int cols, int channels = 1)

Initialization constructor.

Parameters

  • rows: Sensor’s height

  • cols: Sensor’s width

  • channels: Number of channels

MostRecentTimestampBufferT(const MostRecentTimestampBufferT<timestamp_type> &other)

Copy constructor.

Parameters

  • other: The timestamp buffer to construct from

~MostRecentTimestampBufferT()

Destructor.

void create(int rows, int cols, int channels = 1)

Allocates the buffer.

Parameters

  • rows: Sensor’s height

  • cols: Sensor’s width

  • channels: Number of channels

void release()

Deallocates the buffer.

int rows() const

Gets the number of rows of the buffer.

int cols() const

Gets the number of columns of the buffer.

cv::Size size() const

Gets the size of the buffer (i.e. Sensor’s size as well)

int channels() const

Gets the number of channels of the buffer.

bool empty() const

Checks whether the buffer is empty.

void set_to(timestamp_type ts)

Sets all elements of the timestamp buffer to a constant.

Parameters

  • ts: The constant timestamp value

void copy_to(MostRecentTimestampBufferT<timestamp_type> &other) const

Copies this timestamp buffer into another timestamp buffer.

Parameters

  • other: The timestamp buffer to copy to

void swap(MostRecentTimestampBufferT<timestamp_type> &other)

Swaps the timestamp buffer with another one.

Parameters

  • other: The timestamp buffer to swap with

const timestamp_type &at(int y, int x, int c = 0) const

Retrieves a const reference of the timestamp at the specified pixel.

Return

The timestamp at the given pixel

Parameters

  • y: The pixel’s ordinate

  • x: The pixel’s abscissa

  • c: The channel to retrieve the timestamp from

timestamp_type &at(int y, int x, int c = 0)

Retrieves a reference of the timestamp at the specified pixel.

Return

The timestamp at the given pixel

Parameters

  • y: The pixel’s ordinate

  • x: The pixel’s abscissa

  • c: The channel to retrieve the timestamp from

const timestamp_type *ptr(int y = 0, int x = 0, int c = 0) const

Retrieves a const pointer to the timestamp at the specified pixel.

Return

The timestamp at the given pixel

Parameters

  • y: The pixel’s ordinate

  • x: The pixel’s abscissa

  • c: The channel to retrieve the timestamp from

timestamp_type *ptr(int y = 0, int x = 0, int c = 0)

Retrieves a pointer to the timestamp at the specified pixel.

Return

The timestamp at the given pixel

Parameters

  • y: The pixel’s ordinate

  • x: The pixel’s abscissa

  • c: The channel to retrieve the timestamp from

timestamp_type max_across_channels_at(int y, int x) const

Retrieves the maximum timestamp across channels at the specified pixel.

Return

The maximum timestamp at that pixel across all the channels in the buffer

Parameters

  • y: The pixel’s ordinate

  • x: The pixel’s abscissa

void generate_img_time_surface(timestamp_type last_ts, timestamp_type delta_t, cv::Mat &out) const

Generates a CV_8UC1 image of the time surface for the 2 channels.

Side-by-side: negative polarity time surface, positive polarity time surface The time surface is normalized between last_ts (255) and last_ts - delta_t (0)

Parameters

  • last_ts: Last timestamp value stored in the buffer

  • delta_t: Delta time, with respect to last_t, above which timestamps are not considered for the image generation

  • out: The produced image

void generate_img_time_surface_collapsing_channels(timestamp_type last_ts, timestamp_type delta_t, cv::Mat &out) const

Generates a CV_8UC1 image of the time surface, merging the 2 channels.

The time surface is normalized between last_ts (255) and last_ts - delta_t (0)

Parameters

  • last_ts: Last timestamp value stored in the buffer

  • delta_t: Delta time, with respect to last_t, above which timestamps are not considered for the image generation

  • out: The produced image

class Metavision::RateEstimator

Simple estimator class that can estimate average and peak rate online from sample counts.

Public Types

using Callback = std::function<void(timestamp, double, double)>

Callback type to be used in the constructor.

Public Functions

RateEstimator(const Callback &cb = Callback(), timestamp step_time = 100000, timestamp window_time = 1000000)

Constructor.

Parameters

  • cb: Callback that will be called every step_time with the current timestamp, estimated average and peak rates over the counts added in the window time span

  • step_time: Minimum period between two successive callbacks

  • window_time: Time window used to compute the average and peak rates

void add_data(timestamp time, size_t count)

Adds a sample count at t = time.

If the sample count added has a time

time > next_time (next_time = last_callback_time + step_time), then the callback will be called with estimated rates from the available sample counts in the window timespan.
Note

this function must be called with the same time used at last call (the count will then be added to the previous one) or with a time greater than the previous one

Parameters

  • time: Time of the sample

  • count: Count of the sample

void reset_data()

Reset all sample counts.

timestamp step_time() const

Get the minimum time between two callbacks.

Return

The step time

timestamp window_time() const

Get the window duration on which average and peak rates are computed.

Return

The window time

class Metavision::SimpleDisplayer

Class representing a simple displayer.

Public Functions

SimpleDisplayer(const std::string window_name, int fps = 50)

Constructor.

Parameters

  • window_name: Name of the window to display

  • fps: Frames per second to display

~SimpleDisplayer() = default

Default destructor.

void stop()

Quits the display.

void swap_frame(cv::Mat &frame)

Updates the current frame by swap.

Parameters

  • frame: Frame to swap

void copy_frame(const cv::Mat &frame)

Updates the current frame by copy.

Parameters

  • frame: Frame to copy

void set_on_key_pressed_cb(const OnKeyPressedCb &on_key_pressed_cb)

Sets the callback that is called when a key is pressed.

Parameters

  • on_key_pressed_cb: Function to call

void run()

Runs the displayer. Should be called in the main thread.

class Metavision::ThreadedProcess

A convenient object whose purpose is to queue and dequeue tasks in a thread.

Public Functions

~ThreadedProcess()

Destructor.

Waits for all pending tasks completion before leaving the thread. Use abort before destruction to abort the processing.

void add_task(Task task)

Adds a task to the processing queue.

void add_repeating_task(RepeatingTask task)

Adds a task that is repeated once it is done if and only if its result returns true.

bool start()

Starts the processing thread.

Return

false if the processing thread is already started

void stop()

Requests the processing thread to stop and join.

The processing thread remains active until all pending tasks have been processed

void abort()

Requests the processing thread to abort and join.

The threads leaves when it is no longer processing a task. All remaining tasks are dropped.

bool is_active()

Returns if the processing thread is ongoing.