Example of Building a Simple Pipeline

The code sample in <install-prefix>/share/metavision/sdk/core/samples/metavision_filtering will be used here to show how to build a simple pipeline with linear connections. We don’t implement any custom stage in this sample: this example includes only stages that already exist in Metavision SDK.

The sample is aimed at acquiring events data (from a live camera or a RAW file), applying filters on them (ROI and polarity filters), generating a frame out of filtered data and displaying this frame.

The pipeline can be represented by this graph:

The sample includes only the main function that creates the pipeline and connects the stages:

int main(int argc, char *argv[]) {
    std::string in_raw_file_path;

    const std::string short_program_desc("Code sample showing how the pipeline framework can be used to "
                                         "create a simple application to filter and display events.\n");

    const std::string long_program_desc(short_program_desc + "Available keyboard options:\n"
                                                             "  - r - toggle the ROI filter algorithm\n"
                                                             "  - p - show only events of positive polarity\n"
                                                             "  - n - show only events of negative polarity\n"
                                                             "  - a - show all events\n"
                                                             "  - q - quit the application\n");

    po::options_description options_desc("Options");
    // clang-format off
    options_desc.add_options()
        ("help,h", "Produce help message.")
        ("input-raw-file,i", po::value<std::string>(&in_raw_file_path), "Path to input RAW file. If not specified, the camera live stream is used.")
        ;
    // clang-format on

    po::variables_map vm;
    try {
        po::store(po::command_line_parser(argc, argv).options(options_desc).run(), vm);
        po::notify(vm);
    } catch (po::error &e) {
        MV_LOG_ERROR() << short_program_desc;
        MV_LOG_ERROR() << options_desc;
        MV_LOG_ERROR() << "Parsing error:" << e.what();
        return 1;
    }

    if (vm.count("help")) {
        MV_LOG_INFO() << short_program_desc;
        MV_LOG_INFO() << options_desc;
        return 0;
    }

    MV_LOG_INFO() << long_program_desc;

    // A pipeline for which all added stages will automatically be run in their own processing threads (if applicable)
    Metavision::Pipeline p(true);

    // Construct a camera from a recording or a live stream
    Metavision::Camera cam;
    if (!in_raw_file_path.empty()) {
        cam = Metavision::Camera::from_file(in_raw_file_path);
    } else {
        cam = Metavision::Camera::from_first_available();
    }
    const unsigned short width  = cam.geometry().width();
    const unsigned short height = cam.geometry().height();

    /// Pipeline
    //
    //  0 (Camera) -->-- 1 (ROI) -->-- 2 (Polarity) -->-- 3 (Frame Generation) -->-- 4 (Display)
    //

    // 0) Stage producing events from a camera
    auto &cam_stage = p.add_stage(std::make_unique<Metavision::CameraStage>(std::move(cam)));

    // 1) Stage wrapping an ROI filter algorithm
    auto &roi_stage = p.add_algorithm_stage(
        std::make_unique<Metavision::RoiFilterAlgorithm>(80, 80, width - 80, height - 80, false), cam_stage, false);

    // 2) Stage wrapping a polarity filter algorithm
    auto &pol_stage = p.add_algorithm_stage(std::make_unique<Metavision::PolarityFilterAlgorithm>(0), roi_stage, false);

    // 3) Stage generating a frame from filtered events
    auto &frame_stage = p.add_stage(std::make_unique<Metavision::FrameGenerationStage>(width, height, 30), pol_stage);

    // 4) Stage displaying the frame
    auto &disp_stage = p.add_stage(std::make_unique<Metavision::FrameDisplayStage>("CD events"), frame_stage);

    // Run the pipeline step by step to allow user interaction based on key pressed
    while (p.step()) {
        char last_key = disp_stage.get_last_key();
        switch (last_key) {
        case 'a':
            // show all events
            pol_stage.set_enabled(false);
            break;
        case 'n':
            // show only negative events
            pol_stage.set_enabled(true);
            pol_stage.algo().set_polarity(0);
            break;
        case 'p':
            // show only positive events
            pol_stage.set_enabled(true);
            pol_stage.algo().set_polarity(1);
            break;
        case 'r':
            // toggle ROI filter
            roi_stage.set_enabled(!roi_stage.is_enabled());
            break;
        }
    }

    return 0;
}

Instantiating a Pipeline

First, we instantiate the Pipeline using Pipeline::Pipeline(bool auto_detach) constructor:

    Metavision::Pipeline p(true);

We pass auto_detach argument as true to make the pipeline run all stages in their own processing threads.

Adding Stages to the Pipeline

Once the pipeline is instantiated, we add stages to it.

The first stage is a CameraStage, which is used to produce CD events from a camera or a RAW file.

This stage is added to the pipeline by calling Pipeline::add_stage(std::unique_ptr<Stage> &&stage) function. Also, we pass the Camera instance (that is already instantiated in this example) using std::move(cam) to transfer the ownership of the camera smart pointer:

    // 0) Stage producing events from a camera
    auto &cam_stage = p.add_stage(std::make_unique<Metavision::CameraStage>(std::move(cam)));

Next, we add the RoiFilterAlgorithm to apply ROI-filtering and acquire only events in the given Region Of Interest:

    // 1) Stage wrapping an ROI filter algorithm
    auto &roi_stage = p.add_algorithm_stage(
        std::make_unique<Metavision::RoiFilterAlgorithm>(80, 80, width - 80, height - 80, false), cam_stage, false);

Here, we use Pipeline::add_algorithm_stage(std::unique_ptr<Algorithm> &&algo, BaseStage &prev_stage, bool enabled = true) function to add the stage, as we are adding a synchronous algorithm to the pipeline. This stage is deliberately disabled here by setting enabled argument to false. In the main loop, you can see how it can be dynamically activated, by pressing “r” key on keyboard when the sample is running.

Then, we add PolarityFilterAlgorithm to filter events and keep only events of the given polarity. Also this stage is deliberately disabled here by setting enabled argument to false and can be activated dynamically by pressing the “n” or “p” keys on the keyboard (“p” key will keep only events with positive polarity and remove other events, and “n” key will keep only events with negative polarity).

    // 2) Stage wrapping a polarity filter algorithm
    auto &pol_stage = p.add_algorithm_stage(std::make_unique<Metavision::PolarityFilterAlgorithm>(0), roi_stage, false);

Then, we add FrameGenerationStage to generate a frame from events.

    // 3) Stage generating a frame from filtered events
    auto &frame_stage = p.add_stage(std::make_unique<Metavision::FrameGenerationStage>(width, height, 30), pol_stage);

Finally, as the last stage, we add FrameDisplayStage to display the frame on the screen.

    // 4) Stage displaying the frame
    auto &disp_stage = p.add_stage(std::make_unique<Metavision::FrameDisplayStage>("CD events"), frame_stage);

Running the Pipeline

Now, with the set up pipeline, we run the pipeline by calling Pipeline::step() function in a loop:

while (p.step()) {
    // Do any required periodic processing, for example to manage user inputs (e.g. via pressed keyboard keys)
}

In this case, each call of Pipeline::step() function executes a step of the pipeline. Between two calls of Pipeline::step(), the threads for each stage will keep running.