Note

If you are a Prophesee customer, you can access datasheets and user manuals in our Knowledge Center after requesting an account. This page provides additional details on the sensor’s ESP, focusing on how to configure it with the SDK.

Event Signal Processing

Some of the Prophesee sensors provide an Event Signal Processing (ESP) block that provide some event filtering features:

  • Anti-Flicker (AFK) that removes flicker (often considered as noise)

  • Event Trail Filter (STC and Trail) that removes redundant events

  • Event Rate Control (ERC) that adjusts the events spatially and temporally to maintain the event rate below a certain threshold

  • Event Rate Activity Filter that propagates pixel activity only if the event rate is within a certain range.

Note

The SDK allows to store the ESP configuration among all the other camera settings in a JSON file as described in the Saving/loading the camera settings section of the Camera Settings programming guide

Anti-Flicker (AFK)

The anti-flicker function detects and cancels flicker events in 4x4 divided pixel regions on Gen4.1, IMX636 and GenX320 sensors. It is a countermeasure for events generated due to light source with flicker such as fluorescent light. The flicker to be canceled can be set in the range of 2 ms to 20 ms period (50 Hz to 500 Hz).

In Metavision Studio, AFK can be configured (if available on your camera):

AFK In Metavision Studio

The AFK can also be configured using HAL API:

Note

When using the Camera Settings JSON file, the following example illustrates a typical section for configuring the AFK:

{
 "afk_state": {
  "enabled": true,
  "band_low_freq": 50,
  "band_high_freq": 520,
  "filtering_mode": "BAND_STOP",
  "duty_cycle": 50,
  "start_threshold": 6,
  "stop_threshold": 4
 }
}

Note that Metavision SDK CV module also provides a pure software algorithm that is mostly equivalent to this hardware filter:

Event Trail Filter (STC/Trail)

Gen4.1, IMX636 and GenX320 sensors have a filter to cancel redundant information. This filter considers successive events of the same polarity generated in a short time as burst, and will remove some of the events depending on the chose option:

  • Spatio-Temporal-Contrast (STC) retains the second event of a burst. Hence, STC filters out isolated events. STC comes in two versions (STC_CUT_TRAIL and STC_KEEP_TRAIl) that allows to choose if the trail of events after the second one should be removed (CUT) or kept (KEEP).

  • TRAIL retains the first event of a burst after a polarity transition and cancels events of the same polarity that are generated within the Trail threshold period.

Burst event filtering drastically reduces the event rate but it also removes texture information. The time range supported by both filters for treatment as a burst event is 1 ms to 100 ms.

In Metavision Studio, STC and TRAIL can be configured (if available on your camera):

STC/Trail In Metavision Studio

The Event Trail filter can also be configured using HAL API:

Note

When using the Camera Settings JSON file, the following example illustrates a typical section for configuring the Event Trail Filter with TRAIL option:

{
 "event_trail_filter_state": {
  "enabled": true,
  "filtering_type": "TRAIL",
  "threshold": 10000
 }
}

Note that Metavision SDK API provides algorithms that are mostly equivalent to this hardware filter:

Event Rate Controller (ERC)

The ERC performs adjustment by spatially and temporally dropping events to output at the target event rate on Gen4.0, Gen4.1, IMX636 and GenX320 sensors. This filter is located in the final stage of the ESP data processing circuit in order to reduce the event rate after AFK and STC/Trail filtering. It is useful to avoid flooding the computing unit with peaks of events, but keep in mind that signal quality will be degraded when filtering is triggered.

In Metavision Studio, ERC can be configured (if available on your camera):

ERC configuration in Metavision Studio

Note that the Statistic panel shows the ERC drop rate:

ERC drop rate in Metavision Studio

The ERC is also demonstrated in Metavision Viewer and can be configured using HAL API:

Note

When using the Camera Settings JSON file, the following example illustrates a typical section for configuring the ERC at 10 Mev/s on an IMX636-based camera:

{
 "event_rate_control_state": {
  "enabled": true,
  "cd_event_count": 2000
 }
}

The cd_event_count represents the number of events occurring within a defined “count period” (by default: 200us for IMX636 and 100us for GenX320). On IMX636, the target event rate should hence be divided by 5000 to get the cd_event_count Hence, for the IMX636, to calculate the cd_event_count, divide the target event rate by 5000: 10000000 ÷ 5000 = 2000.

Event Rate Activity Filter

Event Rate Activity Filter is a sensor-level filter based only on the event rate. It measures the event rate and propagates pixel activity only if the rate is within a certain range. Note that non-CD events are always preserved (i.e. only “pixel events” are filtered out).

This filter avoids streaming background noise information without relevant activity and streaming high event rate flashing scenes, saving bandwidth and power.

Warning

This filter is currently available only on the GenX320 sensor and is referred to as the Noise Filter (NFL) in the datasheet.

The following diagram shows when CD events are dropped based on the event rate:

Event Rate Activity Filter

Event Rate Activity Filter can be configured using HAL API:

Note

When using the Camera Settings JSON file, the following example illustrates a typical section for configuring the Event Rate Activity Filter with lower thresholds at 1 Mev/s and upper threshold at 10 Mev/s:

{
 "event_rate_activity_filter_state": {
  "enabled": true,
  "lower_start_rate_threshold": 1000000,
  "lower_stop_rate_threshold": 1000000,
  "upper_start_rate_threshold": 10000000,
  "upper_stop_rate_threshold": 10000000
 }
}

Note

In the current SDK release, this filter cannot be configured in Metavision Studio