SDK ML Detection And Tracking API

Generates frames from boxes and tracks

metavision_ml.detection_tracking.display_frame.draw_box_events(frame, box_events, label_map, force_color=None, draw_score=True, thickness=1, color_from='class_id', confidence_field='class_confidence', dic_history={})

Draws boxes on a RGB image.

Parameters

  • frame (np.ndarray) – H, W, 3 image of dtype uint8

  • box_events (np.ndarray) – Box_events in EventBbox format.

  • label_map (list) – List of class names indexed by class_id.

  • force_color (int list) – If not None, overrides the color from color_field and is put as the color

  • boxes. (of all) –

  • draw_score (bool) – Whether to add confidence to the boxes.

  • thickness (int) – Rectangle line thickness.

  • color_from (string) – Field of the EventBbox used to choose the box color, defaults to class_id.

  • confidence_field (string) – Field of the EventBbox used to read confidence score.

  • dic_history (dict) – Dictionary where keys are track_id and values are np.arrays of boxes

Returns

Drawn image.

Return type

img

metavision_ml.detection_tracking.display_frame.draw_detections_and_tracklets(ts, frame, width, height, detections, tracklets, label_map={0: 'background', 1: 'pedestrian', 2: 'two wheeler', 3: 'car'}, list_previous_tracks=[])

Draws a visualization for both detected boxes and tracked boxes side by side.

Detections are on the left pane, with box colors indicating class membership. Tracks are drawn on the right pane with colors denoting track ids.

Parameters

  • ts (int) – Current timestamp in us. (Used for display only).

  • frame (np.ndarray) – Array of dtype np.uint8 and of shape height x width * 2 x 3

  • width (int) – Width of the imager in pixels.

  • height (int) – Height of the imager in pixels.

  • detections (nd.array) – Array of EventBbox to be drawn.

  • tracklets (ndarray) – Array of EventTrackedBox to be drawn.

  • label_map (dict) – Dictionary mapping class ids to the name of the corresponding class

  • list_previous_tracks (list) – list of np.arrays of tracks (one np.array for each previous timestep)

Detection Loader which reads boxes from a CSV file generated by the C++ detection and tracking pipeline

metavision_ml.detection_tracking.io.detections_csv_loader(detections_filename)

This function reloads a text file produced by the c++ detection and tracking pipeline and converts it into a dictionary of numpy structured arrays of EventBbox

Parameters

detections_filename (str) – Filename of detections in the same format as the C++ detection and tracking pipeline

It returns a python dict where the keys are the timestamps. For each timestamp, the associated value is a numpy array of EventBbox for that particular timestamp

Examples

>>> dic_ts_boxes = detections_csv_loader("detections.txt")
>>> np_array_of_all_boxes_detected_at_timestamp_1000 = dic_ts_boxes[1000]
class metavision_ml.detection_tracking.object_detector.ObjectDetector(directory, events_input_width, events_input_height, runtime='cpu', network_input_width=None, network_input_height=None)

Class handling the inference of object detection algorithm.

Parameters

  • directory (string) – Path to a folder containing a model.ptjit network file and a info_ssd_jit.json containing a few hyperparameters.

  • events_input_width (int) – Width of the event sensor used, in pixels.

  • events_input_height (int) – Height of the event sensor used, in pixels.

  • runtime (string) – Corresponds to the torch.device used for computation (“cpu”, “cuda:0”, etc.)

  • network_input_width (int) – Width of the tensor used by the network as input, can be lower than events_input_width to reduce computation.

  • network_input_height (int) – Height of the tensor used by the network as input, can be lower than events_input_height to reduce computation.

cd_proc

Object computing tensor representation from the input events.

Type

object

detection_threshold

Minimal confidence value for a box to be considered.

Type

float

iou_threshold

Minimal Intersection Over Union (IOU) value for a box to be discarded by non maximum suppression.

Type

float

nms_computer_with_rescaling

Object handling Non maximum suppression.

Type

object

get_label_map()

Returns a dict which contains the labels of the classes

get_num_classes()

Gets number of output classes (including background class)

process(ts, frame_buffer_np)

Pass the input frame through the object detector and return the obtained box events.

Parameters

  • ts (int) – Current timestamp in us.

  • frame_buffer_np (np.ndarray) – Input frame buffer.

reset()

Resets the memory cells of the neural network