SDK Calibration Processing

class EBPatternDetector

Base class for pattern detectors using event-based data.

Subclassed by Metavision::calib::ActiveMarkerDetector, Metavision::calib::BinaryChessboardDetector, Metavision::calib::BlinkingChessboardDetector, Metavision::calib::BlinkingDotsGridDetector, Metavision::calib::RectifiedChessboardDetector

Public Functions

EBPatternDetector(const Pattern &pattern)

Constructor.

Parameters

pattern – The pattern to detect

virtual ~EBPatternDetector() = default

Destructor.

virtual std::optional<PatternDetection> detect(const std::vector<Metavision::EventCD> &events) = 0

Detects the pattern in the given events stream.

Parameters

events – The events stream to detect the pattern in

Returns

A pattern detection or nothing if the pattern was not detected

const Pattern &pattern() const

Gets the pattern this detector is looking for.

Returns

The pattern this detector is looking for

class ActiveMarkerDetector : public Metavision::calib::EBPatternDetector

Detector for active markers-based patterns.

Public Functions

ActiveMarkerDetector(const Pattern &pattern, const Params &params = Params())

Constructor.

Parameters

• pattern – The active marker-based pattern to detect

• params – Parameters for the detector

Throws

std::invalid_argument – if:

• the pattern is not of type active marker

• the pattern has less than 4 corners

virtual ~ActiveMarkerDetector() = default

Destructor.

Public Static Attributes

static constexpr const char *const kType = "active-marker"

Type of the detector.

struct Params

Parameters for the ActiveMarkerDetector.

Public Members

ModulatedLightDetectorAlgorithm::Params modulated_light_detector_params

Parameters for the modulated light detector

ActiveMarkerTrackerAlgorithm::Params tracker_params

Parameters for the active marker tracker.

class BinaryChessboardDetector : public Metavision::calib::EBPatternDetector

Chessboard detector that internally works on binary reconstructed frames.

The binary frame is reconstructed by setting pixels corresponding to positive events to 255 and pixels corresponding to negative events to 0. The binary frame is then optionally blurred and morphologically closed to remove noise.

Public Functions

BinaryChessboardDetector(const Pattern &pattern, const Params &params)

Constructor.

Parameters

• pattern – The chessboard pattern to detect

• params – Parameters for the binary chessboard detector

Throws

std::invalid_argument – if the pattern is not of type chessboard

virtual ~BinaryChessboardDetector() = default

Destructor.

Public Static Attributes

static constexpr const char *const kType = "binary-chessboard"

Type of the detector.

struct Params

Parameters for the binary chessboard detector.

Public Members

std::uint32_t sensor_width

Width of the sensor generating the events.

std::uint32_t sensor_height

Height of the sensor generating the events.

std::uint32_t blur_kernel_size = 5

Kernel size for the Gaussian blur, if 0 no blur is applied.

std::uint32_t close_kernel_size = 5

Kernel size for the morphological closing, if 0 no closing is applied.

bool show_binary_frame = false

Whether or not to show the binary frame.

class BlinkingChessboardDetector : public Metavision::calib::EBPatternDetector

Detects a blinking chessboard pattern. It uses a BlinkingFrameGenerator to reconstruct an image of the pattern from the blinking pixels.

Public Functions

BlinkingChessboardDetector(const Pattern &pattern, int sensor_width, int sensor_height, const BlinkingFrameGenerator::Params &params = BlinkingFrameGenerator::Params())

Constructor.

Parameters

• pattern – The chessboard pattern to detect

• sensor_width – The width of the sensor generating the events

• sensor_height – The height of the sensor generating the events

• params – Parameters for the blinking frame generator

Throws

std::invalid_argument – if the pattern is not of type chessboard

virtual ~BlinkingChessboardDetector() = default

Destructor.

Public Static Attributes

static constexpr const char *const kType = "blinking-chessboard"

Type of the detector.

class BlinkingDotsGridDetector : public Metavision::calib::EBPatternDetector

Detects a grid of blinking dots. It uses a frequency-based algorithm (i.e. FrequencyAlgorithm) to detect the blinking pixels from the events stream. The dots are then recovered by clustering the blinking pixels using the FrequencyClusteringAlgorithm. The dots are finally sorted according to the provided grid pattern.

Public Functions

BlinkingDotsGridDetector(const Pattern &pattern, std::uint32_t sensor_width, std::uint32_t sensor_height, const Params &params = Params())

Constructor.

Parameters

• pattern – The blinking dots grid pattern to detect

• sensor_width – The width of the sensor generating the events

• sensor_height – The height of the sensor generating the events

• params – Parameters for the detector

Throws

std::invalid_argument – if the pattern is not of type CirclesGrid

virtual ~BlinkingDotsGridDetector()

Destructor.

Public Static Attributes

static constexpr const char *kType = "blinking-dots-grid"

Type of the detector.

struct Params

Parameters for the BlinkingDotsGridDetector.

Public Members

float special_freq = 125.f

Special frequency, frequency of the first row of the grid, in Hz.

float normal_freq = 166.f

Normal frequency, frequency of the other rows of the grid, in Hz.

timestamp period_diff_thresh_us = 2000

For the frequency estimation, the maximum difference between two successive periods to be considered the same (in us).

int frequency_filter_length = 7

Minimum number of successive stable periods to validate a frequency.

float cluster_center_filter_alpha = 0.05f

Filter constant for the position of the center of the cluster. This value must be > 0 and <=1. Values closer to 1 produce a more reactive, but noisier position estimation.

float max_cluster_frequency_diff = 10.f

Maximum difference to add a frequency event to a cluster, in Hz.

int min_cluster_size = 20

Minimum size of a frequency cluster to be used (in pixels). For a LED grid, larger values can help filtering small clusters produced by reflections, but might prevent detection when the leds are far away.

bool fisheye = false

Allows the detection of strongly distorted grids, as when using a fisheye lens.

class RectifiedChessboardDetector : public Metavision::calib::EBPatternDetector

Special chessboard detector that detects chessboards on rectified images to improve the precision of the detected corners. Chessboards are first detected by an inner chessboard detector, then the detection image is warped to a front-parallel view using a prior camera geometry. The detected corners are then refined in the front-parallel view.

Note

This detector can be used in a two-passes calibration mode where the first pass is used to provide a prior for the camera geometry.

Note

The detection might be slow, so this detector should only be used in offline mode.

Public Functions

RectifiedChessboardDetector(const Params &params)

Constructor.

Parameters

params – The parameters of the detector

Throws

std::invalid_argument – if:

• the sensor width or sensor height is not valid (i.e. 0)

• the blur kernel size is not valid (i.e. 0)

• the chessboard detector is not set

• the camera geometry is not set

• the pattern is not of type chessboard

virtual ~RectifiedChessboardDetector() = default

Destructor.

Public Static Attributes

static constexpr const char *const kType = "rectified-chessboard"

Type of the detector.

struct Params

Parameters for the rectified chessboard detector.

Public Members

std::uint32_t sensor_width

The width of the sensor producing the events.

std::uint32_t sensor_height

The height of the sensor producing the events.

std::uint8_t blur_kernel_size = 11

The size of the kernel used to blur the warped detection images.

bool display_images = false

If true, the warped detection images are displayed.

std::shared_ptr<EBPatternDetector> chessboard_detector

The inner chessboard detector used to detect the chessboard

std::shared_ptr<CameraGeometry32f> camera_geometry

The camera geometry used to warp the detection images.

class BlinkingFrameGenerator

Generates a frame of blinking pixels from an event stream. Positive and negative events are counted independently.

Public Functions

BlinkingFrameGenerator(int sensor_width, int sensor_height, const Params &params = Params())

Constructor.

Parameters

• sensor_width – The width of the sensor generating the events

• sensor_height – The height of the sensor generating the events

• params – Parameters for the blinking frame generator

~BlinkingFrameGenerator() = default

Destructor.

template<typename InputIt>
std::optional<const cv::Mat> process(InputIt it_begin, InputIt it_end)

Processes a range of events and returns the binary frame with blinking pixels if enough blinking pixels.

Template Parameters

InputIt – Type of the iterator

Parameters

• it_begin – Iterator pointing to the beginning of the range of events

• it_end – Iterator pointing to the end of the range of events

Returns

The binary frame with blinking pixels if enough blinking pixels, otherwise an empty optional

struct Params

Parameters for the BlinkingFrameGenerator.

Public Functions

inline Params()

Constructor.

Public Members

int min_num_blinking_pixels = 100

Minimum number of blinking pixels to consider the frame as blinking.

double blinking_pixels_ratios_on = 0.15

Ratio of blinking pixels from positive events to consider the frame as blinking

double blinking_pixels_ratios_off = 0.15

Ratio of blinking pixels from negative events to consider the frame as blinking

int median_blur_radius = 1

Radius of the median blur filter to apply to the positive and negative masks. If negative, no filter is applied

bool enable_event_count = false

Whether or not to count the number of events.

class ExtrinsicsEstimator

Estimates the extrinsics of a set of cameras.

Public Types

using Extrinsics = std::vector<std::pair<cv::Mat, cv::Mat>>

Rotation and translation vectors for each camera.

using Ret = std::pair<ExtrinsicsReport, Extrinsics>

Report and extrinsics.

Public Functions

explicit ExtrinsicsEstimator(std::vector<CameraData> cameras_data)

Constructor.

Parameters

cameras_data – Per camera data needed for the extrinsics estimation

Throws

std::runtime_error – if:

• The number of cameras is less than 2

• The patterns do not have the same number of points

• One of the cameras has no detections

• The number of views in the cameras do not match

• The number of detections in the cameras do not match

virtual ~ExtrinsicsEstimator() = default

Destructor.

std::optional<Ret> calibrate()

Calibrates the extrinsics of the cameras.

Returns

The report and the extrinsics if the calibration succeeded, otherwise an empty optional

struct CameraData

Structure containing the per camera data needed for the extrinsics estimation.

Note

The reference camera is the first camera in the list

Note

The detections are first normalized using the camera geometry before being used for the extrinsics estimation

Public Members

std::shared_ptr<CameraGeometryBase<float>> camera_geometry

Camera geometry (i.e. intrinsic parameters)

Pattern pattern

Pattern detected by this camera.

std::vector<PatternDetection> detections

Pattern detections produced by this camera.

class IntrinsicsEstimator

Base class for intrinsics estimators.

Subclassed by Metavision::calib::PinholeEstimator

Public Functions

IntrinsicsEstimator(std::uint32_t sensor_width, std::uint32_t sensor_height)

Constructor.

Parameters

• sensor_width – Width of the camera sensor

• sensor_height – Height of the camera sensor

virtual ~IntrinsicsEstimator() = default

Destructor.

std::optional<Ret> calibrate(const Pattern &pattern, const std::vector<PatternDetection> &detections)

Calibrates the intrinsics of the camera.

Parameters

• pattern – The pattern used for the calibration

• detections – The detections of the pattern by the camera

Returns

The report and the camera geometry if the calibration succeeded, otherwise an empty optional

class PinholeEstimator : public Metavision::calib::IntrinsicsEstimator

Estimates the intrinsics of a camera using the pinhole model.

Public Functions

PinholeEstimator(const Params &params)

Constructor.

Parameters

params – Parameters needed for the pinhole estimation

Public Static Attributes

static constexpr const char *kType = "pinhole"

Type of the estimator.

struct Params

Structure containing the parameters needed for the pinhole estimation.

Public Members

std::uint32_t sensor_width

Width of the camera sensor.

std::uint32_t sensor_height

Height of the camera sensor.

int flags = cv::CALIB_FIX_ASPECT_RATIO | cv::CALIB_FIX_K3

Flags for the calibration.

double outlier_ths = 2.0

Threshold for the outlier detection.