Note
This C++ sample is available only with our Professional plan. The corresponding Python sample is available with all Metavision Intelligence Plans.
Vibration Estimation using C++
The Analytics API can be used to estimate the frequency of vibrating objects.
The sample metavision_vibration_estimation.cpp
shows how to compute:
a frequency map, that is a per-pixel estimation of the frequency of the received events
the dominant frequency among all pixels, that is the most common frequency.
This sample also allows computing the dominant frequency in ROIs and checking the frequency at specific locations (i.e. pixel by pixel) in the frequency map.
Expected Output
Metavision Vibration Estimation sample visualizes the frequency map (i.e. frequency estimated per pixel), the dominant frequency in the camera’s FOV, and selected ROI(s) with their frequency(ies):
The sample allows the following interactions:
selecting an ROI (with mouse click and drag) for which the dominant frequency will be shown next to the ROI
moving a mouse pointer over vibrating pixels to see frequency pixel by pixel
How to start
First, compile the sample as described in this tutorial.
To start the sample based on the live stream from your camera, run:
Linux
./metavision_vibration_estimation
Windows
metavision_vibration_estimation.exe
To start the sample based on recorded data, provide the full path to a RAW file (here, we use a file from our Sample Recordings):
Linux
./metavision_vibration_estimation -i monitoring_40_50hz.raw
Windows
metavision_vibration_estimation.exe -i monitoring_40_50hz.raw
To check for additional options:
Linux
./metavision_vibration_estimation -h
Windows
metavision_vibration_estimation.exe -h
Code Overview
Pipeline
Metavision Vibration Estimation sample implements the following pipeline:

Frequency Map Async Algorithm Stage
This stage uses the Metavision::FrequencyMapAsyncAlgorithm
to generate a frequency map of the vibrating
objects from the CD events. The Metavision::FrequencyMapAsyncAlgorithm
is asynchronous in the sense that a
frequency map is not produced for each input buffer of events but rather at a fixed refresh rate in the camera’s clock.
So for each input buffer of events, 0, 1 or N frequency map(s) might be produced.
To feed the Metavision::FrequencyMapAsyncAlgorithm
with the input CD events, we specify the consuming
callback of the stage using the Metavision::BaseStage::set_consuming_callback()
method:
set_consuming_callback([this](const boost::any &data) {
try {
auto buffer = boost::any_cast<EventBufferPtr>(data);
if (buffer->empty())
return;
freq_algo_->process_events(buffer->cbegin(), buffer->cend());
} catch (boost::bad_any_cast &c) { MV_LOG_ERROR() << c.what(); }
});
Then, to retrieve the frequency map, we subscribe to the output callback of the
Metavision::FrequencyMapAsyncAlgorithm
and send it to the next stages using the
Metavision::BaseStage::produce()
method:
frequency_map_pool_ = FrequencyMapPool::make_bounded();
freq_algo_->set_output_callback([this](Metavision::timestamp t, FrequencyMap &frequency_map) {
TimedFrequencyMap timed_frequency_map;
timed_frequency_map.first = t;
timed_frequency_map.second = frequency_map_pool_.acquire();
// The frequency map is passed via a non constant reference meaning that we are free to swap it to avoid
// useless copies.
cv::swap(*timed_frequency_map.second, frequency_map);
produce(timed_frequency_map);
});
Note
The fact that the frequency map is passed to the callback via a non constant reference by the
Metavision::FrequencyMapAsyncAlgorithm
allows us cv::swap
it and avoid useless copies. This way the
Metavision::FrequencyMapAsyncAlgorithm
can continue to update the next frequency map while the current
one is sent without any copy to the next stages. Apart from the swap, the frequency map is not modified in the
callback.
Vibration GUI Stage
This stage is in charge of displaying the frequency map in a way that makes it easy for you to visualize the vibrating objects in the camera’s FOV.
Each input frequency map is converted to an RGB frame using the Metavision::HeatMapFrameGeneratorAlgorithm
and a given color map. In addition, the dominant frequency of the scene is computed using the
Metavision::DominantValueMapAlgorithm
and printed in the RGB frame.
Furthermore, the GUI stage allows you to define ROIs in the frequency map by clicking and dragging. The dominant frequency inside that ROI is then automatically computed.
Finally, you can check the frequency pixel by pixel by pointing the mouse cursor at a specific location in the frequency map.
All the previously described processing is done synchronously, meaning that for each input frequency map, an RGB frame is displayed and all the user interactions are handled. For that reason, everything is done the consuming callback of the stage:
set_consuming_callback([this](const boost::any &data) { display_callback(data); });
And the full display callback:
window_->poll_events();
auto ts_frequency_map = boost::any_cast<TimedFrequencyMap>(data);
const auto &input_ts = ts_frequency_map.first;
auto &input_freq_map_ptr = ts_frequency_map.second;
if (!input_freq_map_ptr)
return;
// Generate the heat map
freq_map_display_algo_->generate_bgr_heat_map(*input_freq_map_ptr, display_frame_);
print_dominant_frequency(*input_freq_map_ptr);
print_pixel_frequency(*input_freq_map_ptr);
print_rois(*input_freq_map_ptr);
print_help_message();
window_->show_async(display_frame_, false);
if (window_->should_close())
this->pipeline().cancel();
The following image shows an example of a frequency map displayed in the GUI stage and in which an ROI has been defined:
