SDK Calibration Algorithms

class Metavision::BlinkingDotsGridDetectorAlgorithm

Class that detects a grid of blinking dots with specific frequencies, for example the Prophesee calibration shield.

Public Types

using OutputCb = std::function<void(timestamp, cv::Mat&, std::vector<cv::Point2f>&)>

Type for callback.

Public Functions

BlinkingDotsGridDetectorAlgorithm(int sensor_width, int sensor_height, const BlinkingDotsGridDetectorAlgorithmConfig &config)

Constructor.

Parameters
~BlinkingDotsGridDetectorAlgorithm()

Destructor.

template<typename InputIt>
void process_events(InputIt it_begin, InputIt it_end)

Processes a buffer of events.

Template Parameters

InputIt – Read-Only input event iterator type. Works for iterators over buffers of EventCD or equivalent

Parameters

  • it_begin – Iterator to first input event

  • it_end – Iterator to last input event

void set_output_callback(const OutputCb &output_cb)

Sets a callback to get the detection results. The callback provides:

  • Timestamp of grid detection.

  • Image showing the location of the events used to detect the grid.

  • std::vector with the points in the grid (empty if the grid was not correctly detected).

Note

The generated image and the vector will be passed to the callback as non constant references. The user is free to copy or swap them.

Parameters

output_cb – Function to be set as callback.

class Metavision::BlinkingFrameGeneratorAlgorithm

Class that generates a frame from blinking events.

It accumulates events and keeps pixels which were activated with both polarities during the accumulating period, if enough of them are found. Outputs a binary frame representing blinking pixels (0 or 255).

Public Types

using OutputCb = std::function<void(timestamp, cv::Mat&)>

Type for callbacks called after each asynchronous process.

Public Functions

BlinkingFrameGeneratorAlgorithm(int sensor_width, int sensor_height, const BlinkingFrameGeneratorAlgorithmConfig &config)

Constructor.

Parameters

  • sensor_width – Sensor’s width

  • sensor_height – Sensor’s height

  • config – Blinking detector configuration

template<typename InputIt>
void process_events(InputIt it_begin, InputIt it_end)

Processes a buffer of events.

Template Parameters

InputIt – Read-Only input event iterator type. Works for iterators over buffers of EventCD or equivalent

Parameters

  • it_begin – Iterator to the first input event

  • it_end – Iterator to the past-the-end event

void set_output_callback(const OutputCb &output_cb)

Sets a callback to get the mask.

Note

The generated image will be passed to the callback as a non constant reference, meaning that the user is free to copy it or swap it. In case of a swap with a non initialized image, it will be automatically initialized

class Metavision::DftHighFreqScorerAlgorithm

Class that computes and returns the Discrete Fourier Transform (DFT) of an image in addition to a score depending on the proportion of high frequencies.

Public Functions

DftHighFreqScorerAlgorithm(int width, int height, const DftHighFreqScorerAlgorithmConfig &dft_config)

Constructor.

Parameters

  • width – Sensor’s width (in pixels)

  • height – Sensor’s height (in pixels)

  • dft_config – Discrete Fourier Transform configuration file

bool process_frame(const timestamp ts, const cv::Mat &input_frame, float &output_score)

Computes the DFT and a high frequency score.

Parameters

  • ts – Timestamp

  • input_frame – Binary frame of accumulated blinking events (Either 0 or 1)

  • output_score – Ratio of the average DFT magnitude of high frequencies to the one for all frequencies

Returns

false if it fails

struct Metavision::CalibrationGridPattern

Structure representing the geometry of the rigid 3D pattern used for calibration.

Public Functions

CalibrationGridPattern() = default

Default constructor.

CalibrationGridPattern(unsigned int n_cols, unsigned int n_rows, float dist_between_rows, float dist_between_cols)

Constructor using the physical size of cells.

Parameters

  • n_cols – Number of columns in the pattern (width)

  • n_rows – Number of rows of the pattern (height)

  • dist_between_rows – Distance between two consecutive rows of the pattern

  • dist_between_cols – Distance between two consecutive columns of the pattern

CalibrationGridPattern(unsigned int n_cols, unsigned int n_rows, const std::vector<double> &coordinates_3d)

Constructor using a flattened vector of 3D coordinates.

Parameters

  • n_cols – Number of columns in the pattern (width)

  • n_rows – Number of rows of the pattern (height)

  • coordinates_3d – 3D positions of each point of the pattern, should be n_cols * n_rows * 3 size

CalibrationGridPattern(unsigned int n_cols, unsigned int n_rows, const std::vector<cv::Point3f> &pts_3d)

Constructor using a vector of 3D points.

Parameters

  • n_cols – Number of columns in the pattern (width)

  • n_rows – Number of rows of the pattern (height)

  • pts_3d – 3D points of the pattern, should be n_cols * n_rows size

Public Members

unsigned int n_cols_

Pattern’s width.

unsigned int n_rows_

Pattern’s height.

unsigned int n_pts_

Nbr of points in pattern.

float square_height_ = -1

Distance between two consecutive rows of the pattern.

float square_width_ = -1

Distance between two consecutive columns of the pattern.

std::vector<cv::Point3f> base_3D_points_

3D positions of each point of the pattern

template<typename T>
T Metavision::MonoCalibration::calibrate_opencv(const std::vector<cv::Point3_<T>> &pattern_3d, const std::vector<std::vector<cv::Point_<T>>> &pts_2d, const cv::Size &image_size, MonoCalibration::Model model, int flags, cv::Mat &K, cv::Mat &d, std::vector<bool> *selected_views = nullptr, std::vector<cv::Vec3d> *rvecs = nullptr, std::vector<cv::Vec3d> *tvecs = nullptr, float outlier_ths = 2)

Finds the camera intrinsic and extrinsic parameters from several views of a calibration pattern (using OpenCV)

Parameters

  • pattern_3d – [in] 3D coordinates of the calibration pattern

  • pts_2d – [in] 2D detections of each calibration pattern view

  • image_size – [in] Sensor’s size (in pixels)

  • model – [in] Camera model

  • flags – [in] Calibration flags

  • K – [out] Camera matrix

  • d – [out] Distortion coefficients

  • selected_views – [out] Optional output bitset indicating the views that have been selected for the calibration

  • rvecs – [out] Optional output vector of rotation vectors (Rodrigues) estimated for each pattern view

  • tvecs – [out] Optional output vector of translation vectors estimated for each pattern view

  • outlier_ths – [in] Remove the views for which the reprojection error is more than a certain number of times the standard deviation away from the mean. For instance, a value of 2 means that views with error above (mean+2*std) are removed. A negative threshold can be used to specify that all views must be kept

template<typename T>
void Metavision::MonoCalibration::project_points_opencv(const std::vector<cv::Point3_<T>> &pattern_3d, const cv::Mat &K, const cv::Mat &d, const cv::Vec3d &rvec, const cv::Vec3d &tvec, MonoCalibration::Model model, std::vector<cv::Point_<T>> &projected_pts_2d)

Projects 2D points using camera intrinsic and extrinsic parameters for a given view (using OpenCV)

Parameters

  • pattern_3d – [in] 3D coordinates of the calibration pattern

  • K – [in] Camera matrix

  • d – [in] Distortion coefficients

  • rvec – [in] Rotation vector (Rodrigues) of the input view

  • tvec – [in] Translation vector of the input view

  • model – [in] Camera model corresponding to K and d

  • projected_pts_2d – [out] 2D projections of the pattern from the input view

template<typename T>
void Metavision::MonoCalibration::compute_reprojection_errors_opencv(const std::vector<cv::Point3_<T>> &pattern_3d, const std::vector<std::vector<cv::Point_<T>>> &pts_2d, const cv::Mat &K, const cv::Mat &d, const std::vector<bool> &selected_views, const std::vector<cv::Vec3d> &rvecs, const std::vector<cv::Vec3d> &tvecs, MonoCalibration::Model model, std::vector<float> &rms_errors)

Computes the average reprojection error of each input view given camera intrinsic and extrinsic parameters (using OpenCV)

Parameters

  • pattern_3d – [in] 3D coordinates of the calibration pattern

  • pts_2d – [in] 2D detections of each calibration pattern view

  • K – [in] Camera matrix

  • d – [in] Distortion coefficients

  • selected_views – [int] Bitset indicating the views that been selected for the calibration

  • rvecs – [in] Vector of rotation vectors (Rodrigues) estimated for each pattern view

  • tvecs – [in] Vector of translation vectors estimated for each pattern view

  • model – [in] Camera model corresponding to K and d

  • rms_errors – [out] RMS reprojection errors of each input view