SDK CV Algorithms
-
class
Metavision
::
ActiveMarkerTrackerAlgorithm
A class that tracks an active marker in 2D.
An active marker is a rigid object composed of several blinking LEDs. It is tracked by tracking its LEDs based on the events they generate (i.e. EventSourceId), the latter of which form blobs in the sensor array. The blobs are not directly detected and tracked. Instead, events are associated to tracks using a radius criterion. A monitoring process is then regularly executed to:
re-assess this radius criterion (i.e. proportionally to the lowest distance between the tracks)
remove the tracks that are no longer tracked (i.e. based on an inactivity period, meaning with no update) The first time an event with a valid ID that doesn’t match any existing track is received, a new one is created.
Public Functions
-
ActiveMarkerTrackerAlgorithm
(const Params ¶ms, const std::set<std::uint32_t> &sources_to_track) Constructor.
- Parameters
params – The configuration parameters
sources_to_track – List of unique IDs of the LEDs that constitute the active marker to be tracked
-
template<typename
InputIt
, typenameOutputIt
>
voidprocess_events
(InputIt begin, InputIt end, OutputIt inserter) Tracks the 2D position of the active marker’s LEDs from EventSourceId events.
- Template Parameters
InputIt – Input event iterator type, works for iterators over containers of EventSourceId
OutputIt – Output iterator type, works for iterators over containers of EventActiveTrack
- Parameters
begin – [in] Iterator pointing to the first event in the stream
end – [in] Iterator pointing to the past-the-end element in the stream
inserter – [out] Back inserter
-
struct
Params
Parameters for configuring the ActiveMarkerTrackerAlgorithm.
Public Members
-
bool
update_radius
= true Flag indicating whether to update the radius during the monitoring process.
-
Duration
inactivity_period_us
= 10000 Inactivity period above which a track is considered as lost.
-
float
monitoring_frequency_hz
= 30.f Frequency (in camera time) at which the monitoring process is executed
-
float
radius
= 30.f Influence radius used to associate events to tracks.
-
float
distance_pct
= 0.3f Percentage on the lowest distance between two tracks used to update the radius criterion
-
float
alpha_pos
= 0.01f Influence of a new associated event on the position update of a track.
-
bool
-
template<typename
Use64Bits
= std::true_type>
classMetavision
::
ActivityNoiseFilterAlgorithm
Filter that accepts events if a similar event has happened during a certain time window in the past, in the neighborhood of its coordinates.
Public Functions
-
inline
ActivityNoiseFilterAlgorithm
(std::uint32_t width, std::uint32_t height, timestamp threshold) Builds a new ActivityNoiseFilterAlgorithm object.
- Parameters
width – Maximum X coordinate of the events in the stream
height – Maximum Y coordinate of the events in the stream
threshold – Length of the time window for activity filtering (in us)
-
~ActivityNoiseFilterAlgorithm
() = default Default destructor.
-
template<class
InputIt
, classOutputIt
>
inline OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Applies the Activity Noise filter to the given input buffer storing the result in the output buffer.
- Template Parameters
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
inline timestamp
get_threshold
() Returns the current threshold.
- Returns
Threshold applied while filtering
-
inline
-
class
Metavision
::
AntiFlickerAlgorithm
Algorithm used to remove flickering events given a frequency interval.
Public Functions
-
AntiFlickerAlgorithm
(int width, int height, const FrequencyEstimationConfig &config) Builds a new AntiFlickerAlgorithm object.
- Parameters
width – Sensor’s width
height – Sensor’s height
config – Frequency estimation’s configuration
-
~AntiFlickerAlgorithm
() = default Destructor.
-
bool
set_min_freq
(double min_freq) Sets minimum frequency of the flickering interval.
Note
The value given has to be strictly inferior to maximum frequency
- Parameters
min_freq – Minimum frequency of the flickering interval
- Returns
false if value could not be set (invalid value)
-
bool
set_max_freq
(double max_freq) Sets maximum frequency of the flickering interval.
Note
The value given has to be strictly superior to minimum frequency
- Parameters
max_freq – Maximum frequency of the flickering interval
- Returns
false if value could not be set (invalid value)
-
bool
set_filter_length
(unsigned int filter_length) Sets filter’s length.
- Parameters
filter_length – Number of values in the output median filter
- Returns
false if value could not be set (invalid value)
-
void
set_difference_threshold
(double diff_thresh) Sets the difference allowed between two periods to be considered the same.
- Parameters
diff_thresh – Maximum difference allowed between two successive periods to be considered the same
-
template<class
InputIt
, classOutputIt
>
OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Processes a buffer of events and outputs the non-flickering ones.
- Template Parameters
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
-
class
Metavision
::
DenseFlowFrameGeneratorAlgorithm
Algorithm used to generate visualization images of dense optical flow streams.
Public Types
-
enum class
AccumulationPolicy
Policy for accumulating multiple flow events at a given pixel.
Values:
-
enumerator
Average
Computes the average flow from the observations at the pixel.
-
enumerator
PeakMagnitude
Keeps the highest magnitude flow amongst the observations at the pixel.
-
enumerator
Last
Keeps the most recent flow amongst the observations at the pixel.
-
enumerator
Public Functions
-
DenseFlowFrameGeneratorAlgorithm
(int width, int height, float maximum_flow_magnitude, VisualizationMethod visualization_method = VisualizationMethod::Arrows, AccumulationPolicy accumulation_policy = AccumulationPolicy::Last, int resolution_subsampling = -1) Constructor.
- Parameters
width – Input stream width
height – Input stream height
maximum_flow_magnitude – Scale for highest flow magnitude in the visualization.
visualization_method – Method used to visualize flow field
accumulation_policy – Method used to accumulate multiple flow values at the same pixel.
resolution_subsampling – For Arrows representation, subsampling factor used to accumulate flow events. If negative, accumulation is done at resolution 1/8th. For DenseColorMap representation, this parameter is ignored and flow events are always accumulated in full resolution.
-
template<typename
EventIt
>
voidprocess_events
(EventIt it_begin, EventIt it_end) Processes a buffer of flow events.
Note
Successive calls to process_events will accumulate data at each pixel until generate or reset is called.
- Template Parameters
EventIt – Read-Only input event iterator type. Works for iterators over buffers of EventOpticalFlow or equivalent
- Parameters
it_begin – Iterator to the first input event
it_end – Iterator to the past-the-end event
-
void
generate
(cv::Mat &frame, bool allocate = true) Generates a flow visualization frame.
Note
In DenseColorMap mode, the frame will be reset to zero prior to being filled with the flow visualization. In Arrows mode, the flow visualization will be overlaid on top of the input frame.
- Parameters
frame – Frame that will contain the flow visualization
allocate – Allocates the frame if true. Otherwise, the user must ensure the validity of the input frame. This is to be used when the data ptr must not change (external allocation, ROI over another cv::Mat, …).
- Throws
invalid_argument – if the frame doesn’t have the expected type and geometry
-
void
generate_legend_image
(cv::Mat &legend_frame, int square_size = 0, bool allocate = true) Generates a legend image for the flow visualization.
- Parameters
legend_frame – Frame that will contain the flow visualization legend
square_size – Size of the generated image
allocate – Allocates the frame if true. Otherwise, the user must ensure the validity of the input frame. This is to be used when the data ptr must not change (external allocation, ROI over another cv::Mat, …)
- Throws
invalid_argument – if the frame doesn’t have the expected type
-
void
reset
() Resets the internal states.
-
enum class
-
template<typename
Converter
>
classMetavision
::
EventConverterAlgorithm
This algorithm applies the
()
operator of the Converter class to the input.
-
class
Metavision
::
EventCounterAlgorithm
This class counts the number of events happening during a simulation step.
Public Functions
-
inline
EventCounterAlgorithm
(double threshold = 19) Builds a new EventCounterAlgorithm object.
- Parameters
threshold – Threshold in average rate for the analog_output overlay in Mev/S
-
inline
EventCounterAlgorithm
(const std::string &output_filename, timestamp step_time, timestamp scaling_time, bool enable_log = true, double threshold = 19., bool compute_variance = false) Builds a new EventCounterAlgorithm object.
- Parameters
output_filename – Name of the output file to be generated
step_time – Time interval (in us) used to compute the peak event rate
scaling_time – Time interval (in us) used to compute the average event rate
enable_log – true to save a log file
threshold – Threshold in average rate for the analog_output overlay in Mev/S
compute_variance – true to compute the variance
-
inline
~EventCounterAlgorithm
() Destructor.
-
inline void
enable_log
(bool state) Enables or disables data logging.
- Parameters
state – If true, the logging will start, otherwise it will stop
-
inline void
set_log_destination
(const std::string &csv_file_name) Sets the output file.
-
inline void
set_step_time_us
(timestamp step_time_us) Sets step time.
- Parameters
step_time_us – Time interval (in us) used to compute the peak event rate during the step time (must be smaller than scaling time). Default is 1 ms.
-
inline void
set_scaling_time_us
(timestamp scaling_time_us) Sets scaling time.
- Parameters
scaling_time_us – Time interval (in us) used to compute the average event rate during the scaling time. Default is 1000 ms
-
inline void
polarity_to_count
(int polarity) Counts only the events that have the input polarity.
All events are counted by default. Use -1 to disable the filter after setting a value If the event doesn’t have a polarity, the compilation will fail
- Parameters
polarity – The polarity to count
-
inline void
print_stats
() Prints statistics.
-
inline std::pair<double, double>
get_average_rate
() Gets the average rate last computed.
- Returns
End time (in seconds) of the last average rate that has been computed (first element of the pair) and the average rate (kEv per second) during this interval
-
inline std::pair<double, double>
get_peak_rate
() Gets the peak rate last computed.
- Returns
Time (in seconds) of the last peak rate that has been computed (first element of the pair) and the average peak rate (kEv per second)
-
inline uint64_t
get_events_number
() Gets number of events.
- Returns
Total number of events counted from the input producer since the beginning
-
inline uint64_t
get_events_number_by_polarity
() Gets number of events of the polarity set by polarity_to_count.
- Returns
Total number of events of the polarity set by polarity_to_count counted from the input producer since the beginning
-
inline size_t
add_callback_on_scale
(const std::function<void(double, double)> &cb) Adds a function to be called every time an event rate has been computed.
- Parameters
cb – Function that takes as input the timestamp (in seconds) of the last event the average event rate has been computed on and the average event rate (kEv per second)
- Returns
Callback id
-
inline void
remove_callback_on_scale
(size_t cb_id) Removes the function to call every time an event rate has been computed.
- Parameters
cb_id – Callback id
-
template<typename
InputIt
>
inline std::enable_if_t<!std::is_base_of<Metavision::Event2d, typename Metavision::iterator_traits<InputIt>::value_type>::value>process
(InputIt it, InputIt it_end, timestamp ts) Processes a set of events using iterators.
Note
Two implementations are provided, the first one for events derived from Event2d, the other one for non Event2d-derived events
- Parameters
it – Iterator to the first event
it_end – Iterator to the past-the-end event
ts – Timestamp of current timeslice
-
inline
-
class
Metavision
::
FrequencyAlgorithm
Algorithm used to estimate the flickering frequency (Hz) of the pixels of the sensor.
Public Functions
-
FrequencyAlgorithm
(int width, int height, const FrequencyEstimationConfig &frequency_config) Builds a new FrequencyAlgorithm object.
- Parameters
width – Sensor’s width
height – Sensor’s height
frequency_config – Frequency estimation’s configuration
-
~FrequencyAlgorithm
() Destructor.
-
bool
set_min_freq
(double min_freq) Sets minimum frequency to output.
Note
The value given has to be < maximum frequency
- Parameters
min_freq – Minimum frequency to output
- Returns
false if value could not be set (invalid value)
-
bool
set_max_freq
(double max_freq) Sets maximum frequency to output.
Note
The value given has to be > minimum frequency
- Parameters
max_freq – Maximum frequency to output
- Returns
false if value could not be set (invalid value)
-
bool
set_filter_length
(unsigned int filter_length) Sets filter filter length.
- Parameters
filter_length – Number of values in the output median filter
- Returns
false if value could not be set (invalid value)
-
void
set_difference_threshold
(period_precision diff_thresh) Sets the difference allowed between two periods to be considered the same.
- Parameters
diff_thresh – Maximum difference allowed between two successive periods to be considered the same
-
template<class
InputIt
, classOutputIt
>
voidprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Processes a buffer of events and output inserter for Event2dFrequency<float>
- Template Parameters
InputIt – Read-Only input event iterator type. Works for iterators over buffers of EventCD or equivalent
OutputIt – Read-Write output event iterator type. Works for iterators over containers of Event2dFrequency
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter for Event2dFrequency<float>
-
-
class
Metavision
::
ModulatedLightDetectorAlgorithm
A class that detects modulated lights per pixel.
Modulated light sources encode and transmit their ID by using frequency modulation: symbols can be encoded using the time elapsed between two blinks. Every symbol duration is a multiple of a base period p:
p : logical ‘0’
2p: logical ‘1’
3p: start bit The size of a word (i.e. the number of bits used to encode the ID) is configurable and limited to 32 bits.
Public Functions
-
explicit
ModulatedLightDetectorAlgorithm
(const Params ¶ms) Constructs a ModulatedLightDetectorAlgorithm object with the specified parameters.
- Parameters
params – The parameters for configuring the ModulatedLightDetectorAlgorithm
- Throws
std::invalid_argument – if the resolution is not valid or if the size of a word is greater than 32
-
template<typename
InputIt
, typenameOutputIt
>
OutputItprocess_events
(InputIt begin_it, InputIt end_it, OutputIt output_it) Tries to detect modulated light sources per pixel from events.
One EventSourceId is produced per input EventCD. The id field of an output event will be valid only if the corresponding input event allowed decoding a light ID.
- Template Parameters
InputIt – Input event iterator type
OutputIt – Output iterator type, works for iterators over containers of EventSourceId
- Parameters
begin_it – [in] Iterator pointing to the first event in the stream
end_it – [in] Iterator pointing to the past-the-end element in the stream
output_it – [out] Iterator to the first source id event
- Returns
The iterator to the past-the-last source id event
-
struct
Params
Parameters for configuring the ModulatedLightDetectorAlgorithm.
Public Members
-
std::uint16_t
width
= 0 EB field of view’s width in pixels.
-
std::uint16_t
height
= 0 EB field of view’s height in pixels.
-
std::uint8_t
num_bits
= 8 Size of a word.
-
std::uint32_t
base_period_us
= 200 Base period for encoding bits.
-
float
tolerance
= 0.1f Tolerance used for blink measurement.
-
std::uint16_t
-
class
Metavision
::
PeriodAlgorithm
Algorithm used to estimate the flickering period of the pixels of the sensor.
Public Types
-
using
period_precision
= float Timestamp precision.
Public Functions
-
PeriodAlgorithm
(int width, int height, const PeriodEstimationConfig &config) Builds a new PeriodAlgorithm object.
- Parameters
width – Sensor’s width
height – Sensor’s height
config – period estimation’s configuration
-
~PeriodAlgorithm
() Destructor.
-
bool
set_min_period
(double min_period) Sets minimum period to output.
Note
The value
min_period
has to be smaller than the maximum period- Parameters
min_period – Minimum period (us) to output
- Returns
false if value could not be set (invalid value)
-
bool
set_max_period
(double max_period) Sets maximum period to output.
Note
The value
max_period
has to be larger than the minimum period- Parameters
max_period – Maximum period to output
- Returns
false if value could not be set (invalid value)
-
bool
set_filter_length
(unsigned int filter_length) Sets filter filter length.
- Parameters
filter_length – Number of values in the output median filter
- Returns
false if value could not be set (invalid value)
-
void
set_difference_threshold
(period_precision diff_thresh) Sets the difference allowed between two periods to be considered the same.
- Parameters
diff_thresh – Maximum difference allowed between two successive periods to be considered the same
-
template<class
InputIt
, classOutputIt
>
voidprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Processes a buffer of events and output inserter for Event2dPeriod<T>
- Template Parameters
InputIt – Read-Only input event iterator type. Works for iterators over buffers of EventCD or equivalent
OutputIt – Read-Write output event iterator type. Works for iterators over containers of Event2dPeriod
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter for Event2dPeriod<T>
-
using
-
class
Metavision
::
PlaneFittingFlowAlgorithm
This class is an optimized implementation of the dense optical flow approach proposed in Benosman R., Clercq C., Lagorce X., Ieng S. H., & Bartolozzi C. (2013). Event-based visual flow. IEEE transactions on neural networks and learning systems, 25(2), 407-417.
See also
TripletMatchingFlowAlgorithm algorithm for a more efficient but more noise sensitive dense optical flow approach.
See also
SparseOpticalFlowAlgorithm algorithm for a flow algorithm based on sparse feature tracking, estimating the full scene motion, staged hence more efficient on high event-rate scenes, but also more complex to tune and dependent on the presence of trackable features in the scene.
Note
This dense optical flow approach estimates the flow along the edge’s normal, by fitting a plane locally in the time-surface. The plane fitting helps regularize the estimation, but estimated flow results are still relatively sensitive to noise. The algorithm is run for each input event, generating a dense stream of flow events, but making it relatively costly on high event-rate scenes.
Public Functions
-
inline
PlaneFittingFlowAlgorithm
(int width, int height, int radius = 3, float normalized_flow_magnitude = 100, float min_spatial_consistency_ratio = -1, float max_spatial_consistency_ratio = -1, timestamp fitting_error_tolerance = -1, float neighbor_sample_fitting_fraction = 0.3f) Constructor.
- Parameters
width – Sensor’s width
height – Sensor’s height
radius – Radius used to select timestamps in a time surface around a given location
normalized_flow_magnitude – Normalized magnitude of the optical flow. Pass a negative value or 0 to disable normalization.
min_spatial_consistency_ratio – Lower bound of the acceptable range for the spatial consistency ratio quality indicator. Pass a negative value to disable this test.
max_spatial_consistency_ratio – Upper bound of the acceptable range for the spatial consistency ratio quality indicator. Pass a negative value to disable this test.
fitting_error_tolerance – Tolerance used to accept visual flow estimates with low enough fitting error. Pass a negative value to disable this test.
neighbor_sample_fitting_fraction – Fraction used to determine how many timestamps from the timesurface neighborhood are used to fit the plane.
-
template<typename
InputIt
, typenameOutputIt
>
inline OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Applies the get flow function to the given input buffer storing the result in the output buffer.
- Template Parameters
- Parameters
it_begin – [in] Iterator to first input event
it_end – [in] Iterator to the past-the-end event
inserter – [out] Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
inline
-
class
Metavision
::
ProximityFilterAlgorithm
Class filter that only propagates events close enough to a particular point in the sensor.
Public Functions
-
inline explicit
ProximityFilterAlgorithm
(const Eigen::Vector2f ¢er, float max_distance) Constructor.
- Parameters
center – Point with which the distance is computed
max_distance – Maximal distance to center allowed to keep an event
-
~ProximityFilterAlgorithm
() = default Default destructor.
-
template<class
InputIt
, classOutputIt
>
inline OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Applies the Proximity filter to the given input buffer storing the result in the output buffer.
- Template Parameters
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
inline bool
operator()
(const Event2d &ev) const Basic operator to check if an event is accepted.
- Parameters
ev – Event2D to be tested
-
inline void
set_center
(const Eigen::Vector2f ¢er) Sets the center point of the filter.
- Parameters
center – Point to be used in the filtering process
-
inline void
set_distance
(float max_distance) Sets the maximal allowed distance of the filter.
- Parameters
max_distance – Maximal distance to be used in the filtering process
-
inline Eigen::Vector2f
center
() const Returns the center point used to filter the events.
- Returns
Current reference point used in the filtering process
-
inline float
max_distance
() const Returns the maximal distance used to filter the events.
- Returns
Current maximal distance used in the filtering process
-
inline explicit
-
class
Metavision
::
RoiMaskAlgorithm
Class that only propagates events which are contained in a certain region of interest.
The Region Of Interest (ROI) is defined by a mask (cv::Mat). An event is validated if the mask at the event position stores a positive number.
Alternatively, the user can enable different rectangular regions defined by the upper left corner and the bottom right corner that propagates any event inside them.
Public Functions
-
inline explicit
RoiMaskAlgorithm
(const cv::Mat &pixel_mask) Builds a new RoiMaskAlgorithm object which propagates events in the given window.
- Parameters
pixel_mask – Mask of pixels that should be retained (pixel <= 0 is filtered)
-
template<class
InputIt
, classOutputIt
>
inline OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Applies the ROI Mask filter to the given input buffer storing the result in the output buffer.
- Template Parameters
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
inline void
enable_rectangle
(int x0, int y0, int x1, int y1) Enables a rectangular region defined by the upper left corner and the bottom right corner that propagates any event inside them.
- Parameters
x0 – X coordinate of the upper left corner
y0 – Y coordinate of the upper left corner
x1 – X coordinate of the lower right corner
y1 – Y coordinate of the lower right corner
-
inline int
max_height
() const Returns the maximum number of pixels (height) of the mask.
- Returns
Maximum height of the mask
-
inline int
max_width
() const Returns the maximum number of pixels (width) of the mask.
- Returns
Maximum width of the mask
-
inline const cv::Mat &
pixel_mask
() const Returns the pixel mask of the filter.
- Returns
cv::Mat containing the pixel mask of the filter
-
inline void
set_pixel_mask
(const cv::Mat &mask) Sets the pixel mask of the filter.
- Parameters
mask – Pixel mask to be used while filtering
-
template<typename
T
>
inline booloperator()
(const T &ev) const Basic operator to check if an element is filtered.
- Parameters
ev – Event to check
-
inline bool
operator()
(int x, int y) const Basic operator to check if a position is filtered.
- Parameters
x – X coordinate or the position to check
y – Y coordinate or the position to check
-
inline explicit
-
class
Metavision
::
RotateEventsAlgorithm
class that allows to rotate an event stream.
Note
We assume the rotation to happen with respect to the center of the image
Public Functions
-
inline explicit
RotateEventsAlgorithm
(std::int16_t width_minus_one, std::int16_t height_minus_one, float rotation) Builds a new RotateEventsAlgorithm object with the given width and height.
- Parameters
width_minus_one – Maximum X coordinate of the events (width-1)
height_minus_one – Maximum Y coordinate of the events (height-1)
rotation – Value in radians used for the rotation
-
~RotateEventsAlgorithm
() = default Default destructor.
-
template<class
InputIt
, classOutputIt
>
inline OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Applies the rotate event filter to the given input buffer storing the result in the output buffer.
- Template Parameters
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
inline std::int16_t
width_minus_one
() const Returns the maximum X coordinate of the events.
- Returns
Maximum X coordinate of the events
-
inline void
set_width_minus_one
(std::int16_t width_minus_one) Sets the maximum X coordinate of the events.
- Parameters
width_minus_one – Maximum X coordinate of the events
-
inline std::int16_t
height_minus_one
() const Returns the maximum Y coordinate of the events.
- Returns
Maximum Y coordinate of the events
-
inline void
set_height_minus_one
(std::int16_t height_minus_one) Sets the maximum Y coordinate of the events.
- Parameters
height_minus_one – Maximum Y coordinate of the events
-
inline void
set_rotation
(const float new_angle) Sets the new rotation angle.
- Parameters
new_angle – New angle in rad
-
inline explicit
-
class
Metavision
::
SparseFlowFrameGeneratorAlgorithm
Public Functions
-
template<typename
InputIt
>
inline voidadd_flow_for_frame_update
(InputIt first, InputIt last) Stores one motion arrow per centroid (several optical flow events may have the same centroid) in the motion arrow map to be displayed later using the update_frame_with_flow method.
-
inline void
update_frame_with_flow
(cv::Mat &display_mat) Updates the input frame with the centroids’ motion stored in the history.
Clears the history afterwards
-
template<typename
-
class
Metavision
::
SparseOpticalFlowAlgorithm
Algorithm used to compute the optical flow of objects.
See also
PlaneFittingFlowAlgorithm and TripletMatchingFlowAlgorithm algorithms for simpler methods, with the drawbacks of estimating only the component of the flow along the edge normal and with higher cost on high event-rate scenes.
Note
This sparse optical flow approach tracks small edge-like features and estimates the full motion for these features. This algorithm runs flow estimation only on tracked features, which helps remaining efficient even on high event-rate scenes. However, it requires the presence of trackable features in the scene, and the tuning of the feature detection and tracking stage can be relatively complex.
Public Functions
-
SparseOpticalFlowAlgorithm
(int width, int height, const SparseOpticalFlowConfig &config) Constructor.
- Parameters
width – Sensor’s width
height – Sensor’s height
config – Sparse optical flow’s configuration
-
~SparseOpticalFlowAlgorithm
() Destructor.
-
template<typename
InputIt
, typenameOutputIt
>
OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Applies the optical flow algorithm to the given input buffer storing the result in the output buffer.
Note
the available implementations provided are:
InputIt = std::vector<Event2d>::iterator, OutputIt = std::back_insert_iterator<std::vector<EventOpticalFlow>>
InputIt = std::vector<Event2d>::const_iterator, OutputIt = std::back_insert_iterator<std::vector<EventOpticalFlow>>
InputIt = std::vector<Event2d>::iterator, OutputIt = std::vector<EventOpticalFlow>::iterator
InputIt = std::vector<Event2d>::const_iterator, OutputIt = std::vector<EventOpticalFlow>::iterator
InputIt = const Event2d *, OutputIt = EventOpticalFlow *
- Template Parameters
InputIt – Read-Only input event iterator type
OutputIt – Read-Write output event iterator type
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
-
template<class
itype
>
classMetavision
::
SpatioTemporalContrastAlgorithmT
The SpatioTemporalContrast Filter is a noise filter using the exponential response of a pixel to a change of light to filter out wrong detections and trails.
For an event to be forwarded, it needs to be preceded by another one in a given time window, this ensures that the spatio temporal contrast detection is strong enough. It is also possible to then cut all the following events up to a change of polarity in the stream for that particular pixel (strong trail removal). Note that this will remove signal if 2 following edges of the same polarity are detected (which should not happen that frequently).
Note
The timestamp may be stored in different types 64 bits, 32 bits or 16 bits. The behavior may vary from one size to the other since the number of significant bits may change. Before using the version with less than 32 bits check that the behavior is still valid for the usage.
Public Functions
-
inline
SpatioTemporalContrastAlgorithmT
(int width, int height, timestamp threshold, bool cut_trail = true) Builds a new SpatioTemporalContrast object.
- Parameters
width – Maximum X coordinate of the events in the stream
height – Maximum Y coordinate of the events in the stream
threshold – Length of the time window for filtering (in us)
cut_trail – If true, after an event goes through, it removes all events until change of polarity
-
template<typename
InputIt
, typenameOutputIt
>
inline OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Processes a buffer of events and outputs filtered events.
- Template Parameters
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
inline timestamp
get_threshold
() Returns the threshold for STC filtering.
- Returns
Threshold applied while STC filtering (in us)
-
inline void
set_threshold
(timestamp threshold) Sets the threshold for STC filtering.
- Parameters
threshold – Length of the time window for STC filtering (in us)
-
inline bool
get_cut_trail
() Returns the cut_trail parameter for STC filtering.
- Returns
The cut_trail parameter for STC filtering used for STC filtering
-
inline void
set_cut_trail
(bool v) Sets the cut_trail parameter for STC filtering.
- Parameters
v – If true, after an event goes through, it removes all events until change of polarity
-
inline
-
template<typename
timestamp_type
>
classMetavision
::
TimeGradientFlowAlgorithmT
This class is a local and dense implementation of Optical Flow from events.
Note
This approach is dense in the sense that it processes events at the sensor resolution and produces OpticalFlowEvents potentially on the whole sensor matrix. It computes the optical flow along the edge’s normal by analyzing the recent timestamps at only the left, right, top and down K-pixel far neighbors (i.e. not the whole neighborhood). Thus, the estimated flow results are still quite sensitive to noise. The algorithm is run for each input event, generating a dense stream of flow events, but making it relatively costly on high event-rate scenes. The bit size of the timestamp representation can be reduced to accelerate the processing.
- Template Parameters
timestamp_type – Type of the timestamp used in to compute the optical flow. Typically Metavision::timestamp. Can be used with lighter type (like std::uint32_t) to lower processing time when critical.
Public Functions
-
TimeGradientFlowAlgorithmT
(int input_width, int input_height, const TimeGradientFlowAlgorithmConfig &config) Constructor.
- Parameters
input_width – Maximum width of input events
input_height – Maximum height of input events
config – Configuration for the algorithm
-
template<typename
InputIt
, typenameOutputIt
>
OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Processes each input event to estimate the optical flow from them into the output buffer.
- Template Parameters
InputIt – Read-Only input event iterator type. Works for iterators over buffers of EventCD or equivalent
OutputIt – Read-Write output event iterator type. Works for iterators over containers of EventOpticalFlow or equivalent
- Parameters
it_begin – [in] Iterator to first input event
it_end – [in] Iterator to the past-the-end event
inserter – [out] Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
template<typename
TTimestampType
>
classMetavision
::
TrailFilterAlgorithmT
Filter that accepts an event either if the last event at the same coordinates was of different polarity, or if it happened at least a given amount of time after the last event.
Public Functions
-
inline
TrailFilterAlgorithmT
(uint16_t width, uint16_t height, TTimestampType threshold) Builds a new TrailFilterAlgorithmT object.
- Parameters
width – Maximum X coordinate of the events in the stream
height – Maximum Y coordinate of the events in the stream
threshold – Length of the time window for activity filtering (in us)
-
~TrailFilterAlgorithmT
() = default Default destructor.
-
template<class
InputIt
, classOutputIt
>
inline OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Applies the Trail filter to the given input buffer storing the result in the output buffer.
- Template Parameters
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
inline TTimestampType
get_threshold
() const Returns the current threshold.
- Returns
Threshold applied while filtering
-
inline void
set_threshold
(TTimestampType threshold) Set the current threshold.
- Parameters
threshold – Current threshold
-
inline
-
class
Metavision
::
TransposeEventsAlgorithm
Class that switches X and Y coordinates of an event stream. This filter changes the dimensions of the corresponding frame (width and height are switched)
Public Functions
-
template<class
InputIt
, classOutputIt
>
inline OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Applies the Transpose filter to the given input buffer storing the result in the output buffer.
- Template Parameters
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-
template<class
-
class
Metavision
::
TripletMatchingFlowAlgorithm
This class implements the dense optical flow approach proposed in Shiba S., Aoki Y., & Gallego G. (2022). “Fast Event-Based Optical Flow Estimation by Triplet Matching”. IEEE Signal Processing Letters, 29, 2712-2716.
See also
PlaneFittingFlowAlgorithm algorithm for slightly more accurate but more expensive dense optical flow approach.
See also
SparseOpticalFlowAlgorithm algorithm for a flow algorithm based on sparse feature tracking, estimating the full scene motion, staged hence more efficient on high event-rate scenes, but also more complex to tune and dependent on the presence of trackable features in the scene.
Note
This dense optical flow approach estimates the flow along the edge’s normal, by locally searching for aligned events triplets. The flow is estimated by averaging all aligned event triplets found, which helps regularize the estimates, but results are still relatively sensitive to noise. The algorithm is run for each input event, generating a dense stream of flow events, but making it relatively costly on high event-rate scenes.
Public Functions
-
TripletMatchingFlowAlgorithm
(int width, int height, const Config &config) Constructor.
- Parameters
width – Sensor width
height – Sensor height
config – Configuration for triplet matching flow algorithm, TripletMatchingFlowAlgorithmConfig
-
~TripletMatchingFlowAlgorithm
() Destructor.
-
template<typename
InputIt
, typenameOutputIt
>
OutputItprocess_events
(InputIt it_begin, InputIt it_end, OutputIt inserter) Applies the optical flow algorithm to the given input buffer storing the result in the output buffer.
- Template Parameters
InputIt – Read-Only input event iterator type on EventCD objects
OutputIt – Read-Write output event iterator type on EventOpticalFlow objects
- Parameters
it_begin – Iterator to first input event
it_end – Iterator to the past-the-end event
inserter – Output iterator or back inserter
- Returns
Iterator pointing to the past-the-end event added in the output
-