fiftyone.core.stages.ToFrames

Creates a view that contains one sample per frame in a video collection.

The returned view will contain all frame-level fields and the tags of each video as sample-level fields, as well as a sample_id field that records the IDs of the parent sample for each frame.

By default, sample_frames is False and this method assumes that the frames of the input collection have filepath fields populated pointing to each frame image. Any frames without a filepath populated will be omitted from the returned view.

When sample_frames is True, this method samples each video in the collection into a directory of per-frame images and stores the filepaths in the filepath frame field of the source dataset. By default, each folder of images is written using the same basename as the input video. For example, if frames_patt = "%%06d.jpg", then videos with the following paths:

/path/to/video1.mp4
/path/to/video2.mp4
...

would be sampled as follows:

/path/to/video1/
    000001.jpg
    000002.jpg
    ...
/path/to/video2/
    000001.jpg
    000002.jpg
    ...

However, you can use the optional output_dir and rel_dir parameters to customize the location and shape of the sampled frame folders. For example, if output_dir = "/tmp" and rel_dir = "/path/to", then videos with the following paths:

/path/to/folderA/video1.mp4
/path/to/folderA/video2.mp4
/path/to/folderB/video3.mp4
...

would be sampled as follows:

/tmp/folderA/
    video1/
        000001.jpg
        000002.jpg
        ...
    video2/
        000001.jpg
        000002.jpg
        ...
/tmp/folderB/
    video3/
        000001.jpg
        000002.jpg
        ...

By default, samples will be generated for every video frame at full resolution, but this method provides a variety of parameters that can be used to customize the sampling behavior.

Examples:

import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F

dataset = foz.load_zoo_dataset("quickstart-video")

session = fo.launch_app(dataset)

#
# Create a frames view for an entire video dataset
#

stage = fo.ToFrames(sample_frames=True)
frames = dataset.add_stage(stage)
print(frames)

session.view = frames

#
# Create a frames view that only contains frames with at least 10
# objects, sampled at a maximum frame rate of 1fps
#

num_objects = F("detections.detections").length()
view = dataset.match_frames(num_objects > 10)

stage = fo.ToFrames(max_fps=1)
frames = view.add_stage(stage)
print(frames)

session.view = frames