SDK Core Preprocessors
For a basic usage guide, refer to Preprocessing Events to Tensors.
-
template<typename InputIt>
std::unique_ptr<EventPreprocessor<InputIt>> Metavision::EventPreprocessorFactory::create(const std::unordered_map<std::string, PreprocessingParameters> &proc_params, const TensorShape &tensor_shape) Creates the event processor in adequation with the parameters map describing the event processing to apply.
The map should contain a ‘type’ key, describing the type of preprocessor to create, as well as the necessary parameters for the selected processor. See the different types and required dictionnary parameters below.
A ‘DIFF’ instance requires:
max_incr_per_pixel (float)
clip_value_after_normalization (float)
scale_width (float)
scale_height (float)
A ‘HISTO’ instance requires:
max_incr_per_pixel (float)
clip_value_after_normalization (float)
use_CHW (bool)
scale_width (float)
scale_height (float)
An ‘EVENT_CUBE’ instance requires:
delta_t (timestamp)
max_incr_per_pixel (float)
clip_value_after_normalization (float)
num_utbins (int)
split_polarity (bool)
scale_width (float)
scale_height (float)
A ‘HARDWARE_DIFF’ instance requires:
min_val (int8_t)
max_val (int8_t)
allow_rollover (bool)
A ‘HARDWARE_HISTO’ instance requires:
neg_saturation (uint8_t)
pos_saturation (uint8_t)
allow_rollover (bool)
A ‘TIME_SURFACE’ instance requires:
nb_channels (uint8_t)
- Template Parameters
InputIt – The type of the input iterator for the range of events to process
- Parameters
proc_params – A dictionnary containing parameters describing the processor to instantiate
tensor_shape – Shape of the tensor to fill with preprocessed data (it must match the dimensions of the events which will be passed to the processor). It is expected to provide at least “H” and “W” dimensions.
-
template<typename InputIt>
class EventPreprocessor Processes events to update data from a tensor.
This is the base class. It handles the rescaling of the events if necessary. It also provides accessors to get the shape of the output tensor. Derived class implement the computation. Calling process_events() on this base class triggers the computation to update the provided tensor. This tensor can typically be used as input of a neural network.
- Template Parameters
InputIt – The type of the input iterator for the range of events to process
Subclassed by Metavision::DiffProcessor< InputIt >, Metavision::EventCubeProcessor< InputIt >, Metavision::HardwareDiffProcessor< InputIt >, Metavision::HardwareHistoProcessor< InputIt >, Metavision::HistoProcessor< InputIt >, Metavision::TimeSurfaceProcessor< InputIt, CHANNELS >
Public Functions
-
const TensorShape &get_output_shape() const
Retrieves the shape of the processor’s output tensor. This shape can be used to initialize the output tensor provided to the process_events method.
- Returns
The output tensor shape
-
BaseType get_output_type() const
Retrieves the type of the processor’s output tensor This type can be used to initialize the output tensor provided to the process_events method.
- Returns
The output tensor type
-
void process_events(const timestamp cur_frame_start_ts, InputIt begin, InputIt end, Tensor &tensor) const
Updates the output tensor depending on the input events.
Warning
The tensor needs to have its memory already allocated, which can be done thanks to the class get_output_shape and get_output_type methods and the Tensor method Tensor::create.
- Parameters
cur_frame_start_ts – [in] starting timestamp of the current frame
begin – [in] Begin iterator
end – [in] End iterator
tensor – [out] Updated output tensor
-
template<typename InputIt>
class EventCubeProcessor : public Metavision::EventPreprocessor<InputIt> Class used to compute an event cube from a stream of EventCD.
- Template Parameters
InputIt – The type of the input iterator for the range of events to process
Public Functions
-
EventCubeProcessor(timestamp delta_t, int event_input_width, int event_input_height, int num_utbins, bool split_polarity, float max_incr_per_pixel, float clip_value_after_normalization = 0.f, float width_scale = 1.f, float height_scale = 1.f)
Constructor.
- Parameters
delta_t – Delta time used to accumulate events inside the frame
event_input_width – Width of the event stream
event_input_height – Height of the event stream
num_utbins – Number of micro temporal bins
split_polarity – Process positive and negative events into separate channels
max_incr_per_pixel – Maximum number of increments per pixel. This is used to normalize the contribution of each event
clip_value_after_normalization – Clipping value to apply after normalization (typically: 1.)
width_scale – Scale on the width previously applied to input events. This factor is considered to modulate the contribution of each event at its coordinates.
height_scale – Scale on the height previously applied to input events. This factor is considered to modulate the contribution of each event at its coordinates.
-
template<typename InputIt>
class DiffProcessor : public Metavision::EventPreprocessor<InputIt> Class used to compute the diff image from a stream of EventCD.
- Template Parameters
InputIt – The type of the input iterator for the range of events to process
Public Functions
-
DiffProcessor(int event_input_width, int event_input_height, float max_incr_per_pixel, float clip_value_after_normalization, float width_scale = 1.f, float height_scale = 1.f)
Constructor.
- Parameters
event_input_width – Maximum width of input events
event_input_height – Maximum height of input events
max_incr_per_pixel – Maximum number of increments per pixel. This is used to normalize the contribution of each event
clip_value_after_normalization – Clipping value to apply after normalization (typically: 1.)
width_scale – Scale on the width previously applied to input events. This factor is considered to modulate the contribution of each event at its coordinates.
height_scale – Scale on the height previously applied to input events. This factor is considered to modulate the contribution of each event at its coordinates.
-
template<typename InputIt>
class HardwareDiffProcessor : public Metavision::EventPreprocessor<InputIt> Updates the data from a diff event frame in the form of a tensor with an input stream of events.
This class implements the algorithm for producing diff event frames from a stream of events. This algorithm computes, at each pixel, the sum of polarities of all processed events. Generated event frames are stored using row-major convention.
- Template Parameters
InputIt – The type of the input iterator for the range of events to process.
Public Functions
-
HardwareDiffProcessor(int width, int height, int8_t min_val, int8_t max_val, bool allow_rollover = true)
Constructor.
- Parameters
width – Width of the event stream
height – Height of the event stream
min_val – Lower representable value
max_val – Higher representable value
allow_rollover – If true, a roll-over will be realized when reaching minimal or maximal value. Else, the pixel value will be saturated.
-
void process_events(InputIt begin, InputIt end, RawEventFrameDiff &diff) const
Updates the provided diff frame with the input events.
- Parameters
begin – [in] Iterator pointing to the beginning of the events buffer
end – [in] Iterator pointing to the end of the events buffer
diff – [out] Difference frame to update
-
template<typename InputIt>
class HistoProcessor : public Metavision::EventPreprocessor<InputIt> Class used to compute a histogram from a stream of EventCD.
- Template Parameters
InputIt – The type of the input iterator for the range of events to process
Public Functions
-
HistoProcessor(int event_input_width, int event_input_height, float max_incr_per_pixel, float clip_value_after_normalization, bool use_CHW = true, float width_scale = 1.f, float height_scale = 1.f)
Constructor.
- Parameters
event_input_width – Maximum width of input events
event_input_height – Maximum height of input events
max_incr_per_pixel – Maximum number of increments per pixel. This is used to normalize the contribution of each event
clip_value_after_normalization – Clipping value to apply after normalization (typically: 1.)
use_CHW – Boolean to define frame dimension order, True if the fields’ frame order is (Channel, Height, Width)
width_scale – Scale on the width previously applied to input events. This factor is considered to modulate the contribution of each event at its coordinates.
height_scale – Scale on the height previously applied to input events. This factor is considered to modulate the contribution of each event at its coordinates.
-
template<typename InputIt>
class HardwareHistoProcessor : public Metavision::EventPreprocessor<InputIt> Updates an input histogram with an input stream of events.
This algorithm updates, separately at each pixel, the sum of positive and negative events from the event stream. The histogram values will saturate if more events than the maximum representable integers are received at a given pixel for a given polarity. This algorithm expects a pre-allocated tensor using row-major convention with interleaved channels for each polarity.
- Template Parameters
InputIt – The type of the input iterator for the range of events to process.
Public Functions
-
HardwareHistoProcessor(int width, int height, uint8_t neg_saturation = 255, uint8_t pos_saturation = 255)
Constructor.
- Parameters
width – Width of the event stream
height – Height of the event stream
neg_saturation – Maximum value for the count of negative events in the histogram at each pixel
pos_saturation – Maximum value for the count of positive events in the histogram at each pixel
-
void process_events(InputIt begin, InputIt end, RawEventFrameHisto &histo) const
Updates the provided histogram with the input events.
- Parameters
begin – [in] Iterator pointing to the beginning of the events buffer
end – [in] Iterator pointing to the end of the events buffer
histo – [out] Histogram to update
-
template<typename InputIt, int CHANNELS = 1>
class TimeSurfaceProcessor : public Metavision::EventPreprocessor<InputIt> Class that produces a MostRecentTimestampBuffer (a.k.a. time surface) from events.
- Template Parameters
InputIt – The type of the input iterator for the range of events to process
CHANNELS – Number of channels to use for producing the time surface. Only two values are possible for now: 1 or 2. When a 1-channel time surface is used, events with different polarities are stored all together while they are stored separately when using a 2-channels time surface.
Public Functions
-
TimeSurfaceProcessor(int width, int height)
Constructs a new time surface producer.
- Parameters
width – Sensor’s width
height – Sensor’s height
-
void process_events(InputIt begin, InputIt end, MostRecentTimestampBuffer &time_surface) const
Updates the provided time surface with the input events.
- Parameters
begin – [in] Iterator pointing to the beginning of the events buffer
end – [in] Iterator pointing to the end of the events buffer
time_surface – [out] Time surface to update
-
class Tensor
Generic class to store information about an N-dimension tensor and its content.
Public Functions
-
Tensor()
Default constructor, no memory allocation.
-
Tensor(const TensorShape &shape, const BaseType &type)
Constructor with no input data Memory is allocated with respect to provided shape ad data type. Tensor values are initialized to 0.
- Parameters
shape – The shape of the tensor
type – The type of data stored in the tensor
-
Tensor(const TensorShape &shape, const BaseType &type, std::byte *ptr, bool copy = false)
Main constructor for the tensor class.
- Parameters
shape – The shape of the tensor
type – The type of data stored in the tensor
ptr – Pointer to the tensor data (use reinterpret_cast if necessary). The ownership of the pointer is never transferred.
copy – Indicates whether or not to copy the tensor data. If false, the pointer will be used directly and assumed to be valid.
-
Tensor &operator=(const Tensor &other)
Copy assignment operator.
- Parameters
other – Instance to copy from
-
Tensor &operator=(Tensor &&other) noexcept
Move assignment operator.
- Parameters
other – instance to move from
-
void create(const TensorShape &shape, const BaseType &type, std::byte *ptr, bool copy = false)
Updates the tensor with new shape, type and data.
- Parameters
shape – New shape of the tensor
type – New type of the tensor’s data
ptr – Pointer to the new tensor’s data
copy – If true, the data pointed by ptr will be copied internally. If the new required memory is equal or lower than the previous one, no memory is reallocated.
-
void create(const TensorShape &shape, const BaseType &type)
Allocates memory to the tensor for a given shape and type of data The tensor values are set to 0 and can be set later on thanks to the Tensor::data() accessor.
- Parameters
shape – New shape of the tensor
type – New type of the tensor’s data
-
TensorShape shape() const
Gets the shape of the tensor.
- Returns
The tensor shape
-
BaseType type() const
Gets the base type of the data stored in the tensor.
- Returns
The base type of the tensor’s data
-
size_t byte_size() const
Returns the amount of bytes necessary to store the tensor data.
- Returns
The tensor’s data size in bytes
-
bool empty()
Indicates whether the tensor contains any data.
- Returns
True if the tensor is empty (contains 0 values)
-
template<typename T = std::byte>
const T *data() const Gets the tensor data as const variable.
- Returns
A const pointer to the tensor data
-
template<typename T = std::byte>
T *data() Gets the tensor data.
- Returns
A pointer to the tensor data
-
Tensor()