SDK Core ML Losses API

Perceptual Loss

VGG Perceptual Loss (pretrained, uncalibrated)

class metavision_core_ml.losses.perceptual_loss.VGGPerceptualLoss(resize=True, freeze_bn=False)

VGG Perceptual Loss, it computes L1 distance between several layers of activations between 2 images.

Parameters

resize (bool) – resize to original size before running the loss

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(input, target)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

train(mode: bool)

Set the module in training mode.

This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. Dropout, BatchNorm, etc.

Parameters

mode (bool) – whether to set training mode (True) or evaluation mode (False). Default: True.

Returns

self

Return type

Module

Warping functions

Pytorch Functions for warping an image using the flow.

metavision_core_ml.losses.warp.backwarp_2d(source, y_displacement, x_displacement)

Returns warped source image and occlusion_mask. Value in location (x, y) in output image in taken from (x + x_displacement, y + y_displacement) location of torch source image. If torch location in the source image is outside of its borders, torch location in the target image is filled with zeros and the location is added to torch “occlusion_mask”.

Parameters

  • source – is a tensor witorch indices [example_index, channel_index, y, x].

  • x_displacement –

:param : :param y_displacement: are tensors witorch indices [example_index,

1, y, x].

Returns

is a tensor witorch indices

[example_index, channel_index, y, x].

occlusion_mask: is a tensor witorch indices [example_index, 1, y, x].

Return type

target

metavision_core_ml.losses.warp.compute_source_coordinates(y_displacement, x_displacement)

Returns source coordinates, given displacements.

Given target coordinates (y, x), torch source coordinates are computed as (y + y_displacement, x + x_displacement).

Parameters

  • x_displacement – are tensors witorch indices [example_index, 1, y, x]

  • y_displacement – are tensors witorch indices [example_index, 1, y, x]

metavision_core_ml.losses.warp.create_meshgrid(width, height, is_cuda)

Creates a 2d grid of X-Y coordinates.

Parameters

  • width (int) – desired width

  • height (int) – desired height

  • is_cuda (bool) – device on gpu

metavision_core_ml.losses.warp.ssl_flow_l1(img_seq, flow_seq, direction='backward')

The flow with meshgrid + flow convention either: - warp backward if flow is flow forward flow_t->t+1: gt is future - warp forward if flow is flow backward flow_t->t-1: gt is past

Parameters

  • img_source – (T,B,C,H,W)

  • flow_seq – (T,B,C,H,W)