SDK Calibration Algorithms

class BlinkingFrameGeneratorAlgorithm

Class that generates a frame from blinking events.

It accumulates events and keeps pixels which were activated with both polarities during the accumulating period, if enough of them are found. Outputs a binary frame representing blinking pixels (0 or 255).

Public Types

using OutputCb = std::function<void(timestamp, cv::Mat&)>

Type for callbacks called after each asynchronous process.

Public Functions

BlinkingFrameGeneratorAlgorithm(int sensor_width, int sensor_height, const BlinkingFrameGeneratorAlgorithmConfig &config)

Constructor.

Parameters

  • sensor_width – Sensor’s width

  • sensor_height – Sensor’s height

  • config – Blinking detector configuration

template<typename InputIt>
void process_events(InputIt it_begin, InputIt it_end)

Processes a buffer of events.

Template Parameters

InputIt – Read-Only input event iterator type. Works for iterators over buffers of EventCD or equivalent

Parameters

  • it_begin – Iterator to the first input event

  • it_end – Iterator to the past-the-end event

void set_output_callback(const OutputCb &output_cb)

Sets a callback to get the mask.

Note

The generated image will be passed to the callback as a non constant reference, meaning that the user is free to copy it or swap it. In case of a swap with a non initialized image, it will be automatically initialized

class DftHighFreqScorerAlgorithm

Class that computes and returns the Discrete Fourier Transform (DFT) of an image in addition to a score depending on the proportion of high frequencies.

Public Functions

DftHighFreqScorerAlgorithm(int width, int height, const DftHighFreqScorerAlgorithmConfig &dft_config)

Constructor.

Parameters

  • width – Sensor’s width (in pixels)

  • height – Sensor’s height (in pixels)

  • dft_config – Discrete Fourier Transform configuration file

bool process_frame(const timestamp ts, const cv::Mat &input_frame, float &output_score)

Computes the DFT and a high frequency score.

Parameters

  • ts – Timestamp

  • input_frame – Binary frame of accumulated blinking events (Either 0 or 1)

  • output_score – Ratio of the average DFT magnitude of high frequencies to the one for all frequencies

Returns

false if it fails

template<typename T>
double Metavision::MonoCalibration::calibrate_opencv(const std::vector<cv::Point3_<T>> &pattern_3d, const std::vector<std::vector<cv::Point_<T>>> &pts_2d, const cv::Size &image_size, MonoCalibration::Model model, int flags, cv::Mat &K, cv::Mat &d, std::vector<bool> *selected_views = nullptr, std::vector<cv::Vec3d> *rvecs = nullptr, std::vector<cv::Vec3d> *tvecs = nullptr, double outlier_ths = 2.0)

Finds the camera intrinsic and extrinsic parameters from several views of a calibration pattern (using OpenCV)

Parameters

  • pattern_3d – [in] 3D coordinates of the calibration pattern

  • pts_2d – [in] 2D detections of each calibration pattern view

  • image_size – [in] Sensor’s size (in pixels)

  • model – [in] Camera model

  • flags – [in] Calibration flags

  • K – [out] Camera matrix

  • d – [out] Distortion coefficients

  • selected_views – [out] Optional output bitset indicating the views that have been selected for the calibration

  • rvecs – [out] Optional output vector of rotation vectors (Rodrigues) estimated for each pattern view

  • tvecs – [out] Optional output vector of translation vectors estimated for each pattern view

  • outlier_ths – [in] Remove the views for which the reprojection error is more than a certain number of times the standard deviation away from the mean. For instance, a value of 2 means that views with error above (mean+2*std) are removed. A negative threshold can be used to specify that all views must be kept

Returns

The overall RMS re-projection error

template<typename T>
void Metavision::MonoCalibration::project_points_opencv(const std::vector<cv::Point3_<T>> &pattern_3d, const cv::Mat &K, const cv::Mat &d, const cv::Vec3d &rvec, const cv::Vec3d &tvec, MonoCalibration::Model model, std::vector<cv::Point_<T>> &projected_pts_2d)

Projects 2D points using camera intrinsic and extrinsic parameters for a given view (using OpenCV)

Parameters

  • pattern_3d – [in] 3D coordinates of the calibration pattern

  • K – [in] Camera matrix

  • d – [in] Distortion coefficients

  • rvec – [in] Rotation vector (Rodrigues) of the input view

  • tvec – [in] Translation vector of the input view

  • model – [in] Camera model corresponding to K and d

  • projected_pts_2d – [out] 2D projections of the pattern from the input view

template<typename T>
void Metavision::MonoCalibration::compute_reprojection_errors_opencv(const std::vector<cv::Point3_<T>> &pattern_3d, const std::vector<std::vector<cv::Point_<T>>> &pts_2d, const cv::Mat &K, const cv::Mat &d, const std::vector<bool> &selected_views, const std::vector<cv::Vec3d> &rvecs, const std::vector<cv::Vec3d> &tvecs, MonoCalibration::Model model, std::vector<double> &rms_errors)

Computes the average reprojection error of each input view given camera intrinsic and extrinsic parameters (using OpenCV)

Parameters

  • pattern_3d – [in] 3D coordinates of the calibration pattern

  • pts_2d – [in] 2D detections of each calibration pattern view

  • K – [in] Camera matrix

  • d – [in] Distortion coefficients

  • selected_views – [int] Bitset indicating the views that been selected for the calibration

  • rvecs – [in] Vector of rotation vectors (Rodrigues) estimated for each pattern view

  • tvecs – [in] Vector of translation vectors estimated for each pattern view

  • model – [in] Camera model corresponding to K and d

  • rms_errors – [out] RMS reprojection errors of each input view

double CeresCalibration::calibrate_n_cameras(const std::vector<std::vector<cv::Mat>> &object_points, const std::vector<std::vector<cv::Mat>> &image_points, const cv::Mat &extrinsic_ids, std::vector<cv::Mat> &camera_matrices, std::vector<cv::Mat> &dist_coeffs, std::vector<cv::Mat> &rotation_vectors, std::vector<cv::Mat> &translation_vectors, boost::optional<std::vector<cv::Mat>&> per_view_rvecs, boost::optional<std::vector<cv::Mat>&> per_view_tvecs, boost::optional<std::vector<cv::Mat>&> std_devs_intrinsics, boost::optional<cv::Mat&> std_devs_extrinsics, boost::optional<std::vector<cv::Mat>&> per_view_errors, unsigned long long flags, cv::TermCriteria termination_criteria, bool log_iterations = false)

Camera calibration function for n cameras. Calibrates n cameras, given a object points in n_view, and image points in n_cameras x n_views. Optimizes both intrinsic parameters of each camera and extrinsic parameters of each extrinsic id. Correspondence between cameras and extrinsic ids are given in extrinsic_ids.

Parameters

  • object_points – [in] List of list of array of 3d points in world coordinates

  • image_points – [in] List of list of array of 2d points in pixel coordinates

  • extrinsic_ids – [in] Array of ints, id of camera extrinsic for each camera

  • camera_matrices – [inout] Camera matrix of cameras

  • dist_coeffs – [inout] Distortion coeffs of cameras

  • rotation_vectors – [inout] Rotation vector from camera 1 frame to camera i frame for each extrinsic index i

  • translation_vectors – [inout] Translation from camera 1 frame to camera i frame for each extrinsic index i

  • per_view_rvecs – [out] Rotation vector from pattern to camera 1 frame in each view

  • per_view_tvecs – [inout] Translation from from pattern to camera 1 frame in each view. Used as input to set pattern distance

  • std_devs_intrinsics – [in] Standard deviation of intrinsics for each camera

  • std_devs_extrinsics – [inout] Standard deviation of camera pose

  • per_view_errors – [out] std::vector of per_view_errors (one matrix of size n_images per camera)

  • flags – [in] OpenCV calibration flags

  • termination_criteria – [in] Optimization termination criteria

  • log_iterations – [in] Print ceres iterations

Returns

Average reprojection error