fiftyone.core.models.Model

class documentation

class Model(etal.Model): (source)

Known subclasses: fiftyone.utils.eta.ETAModel, fiftyone.utils.sam2.SegmentAnything2VideoModel, fiftyone.utils.torch.TorchImageModel, fiftyone.utils.transformers.FiftyOneTransformer, fiftyone.utils.transformers.FiftyOneZeroShotTransformer, fiftyone.utils.ultralytics.FiftyOneRTDETRModel, fiftyone.utils.ultralytics.FiftyOneYOLOModel

View In Hierarchy

Abstract base class for models.

This class declares the following conventions:

Model.__init__ should take a single config argument that is an instance of <Model>Config
Models implement the context manager interface. This means that models can optionally use context to perform any necessary setup and teardown, and so any code that builds a model should use the with syntax

Method	`__enter__`	Undocumented
Method	`__exit__`	Undocumented
Method	`predict`	Performs prediction on the given data.
Method	`predict_all`	Performs prediction on the given iterable of data.
Method	`preprocess.setter`	Undocumented
Property	`can_embed_prompts`	Whether this instance can generate prompt embeddings.
Property	`has_embeddings`	Whether this instance can generate embeddings.
Property	`has_logits`	Whether this instance can generate logits for its predictions.
Property	`media_type`	The media type processed by the model.
Property	`preprocess`	Whether to apply `transforms` during inference (True) or to assume that they have already been applied (False).
Property	`ragged_batches`	True/False whether `transforms` may return tensors of different sizes. If True, then passing ragged lists of data to `predict_all` is not allowed.
Property	`transforms`	The preprocessing function that will/must be applied to each input before prediction, or `None` if no preprocessing is performed.

def __enter__(self): (source) ¶

overridden in fiftyone.utils.eta.ETAModel, fiftyone.utils.torch.TorchImageModel

Undocumented

def __exit__(self, *args): (source) ¶

overridden in fiftyone.utils.eta.ETAModel, fiftyone.utils.torch.TorchImageModel

Undocumented

def predict(self, arg): (source) ¶

overridden in fiftyone.utils.eta.ETAModel, fiftyone.utils.sam2.SegmentAnything2VideoModel, fiftyone.utils.torch.TorchImageModel, fiftyone.utils.transformers.FiftyOneTransformer, fiftyone.utils.transformers.FiftyOneZeroShotTransformer, fiftyone.utils.ultralytics.FiftyOneRTDETRModel, fiftyone.utils.ultralytics.FiftyOneYOLOModel

Performs prediction on the given data.

Image models should support, at minimum, processing arg values that are uint8 numpy arrays (HWC).

Video models should support, at minimum, processing arg values that are eta.core.video.VideoReader instances.

Parameters
arg	the data
Returns
a `fiftyone.core.labels.Label` instance or dict of `fiftyone.core.labels.Label` instances containing the predictions

def predict_all(self, args): (source) ¶

overridden in fiftyone.utils.eta.ETAModel, fiftyone.utils.torch.TorchImageModel, fiftyone.utils.transformers.FiftyOneTransformerForDepthEstimation, fiftyone.utils.transformers.FiftyOneTransformerForImageClassification, fiftyone.utils.transformers.FiftyOneTransformerForObjectDetection, fiftyone.utils.transformers.FiftyOneTransformerForSemanticSegmentation, fiftyone.utils.transformers.FiftyOneZeroShotTransformerForImageClassification, fiftyone.utils.ultralytics.FiftyOneYOLODetectionModel, fiftyone.utils.ultralytics.FiftyOneYOLOOBBModel, fiftyone.utils.ultralytics.FiftyOneYOLOPoseModel

Performs prediction on the given iterable of data.

Image models should support, at minimum, processing args values that are either lists of uint8 numpy arrays (HWC) or numpy array tensors (NHWC).

Video models should support, at minimum, processing args values that are lists of eta.core.video.VideoReader instances.

Subclasses can override this method to increase efficiency, but, by default, this method simply iterates over the data and applies predict to each.

Parameters
args	an iterable of data
Returns
a list of `fiftyone.core.labels.Label` instances or a list of dicts of `fiftyone.core.labels.Label` instances containing the predictions

@preprocess.setter
def preprocess(self, value): (source) ¶

overridden in fiftyone.utils.eta.ETAModel, fiftyone.utils.torch.TorchImageModel

Undocumented

@property
can_embed_prompts = (source) ¶

overridden in fiftyone.utils.clip.zoo.TorchCLIPModel, fiftyone.utils.open_clip.TorchOpenClipModel

Whether this instance can generate prompt embeddings.

This method returns False by default. Methods that can generate prompt embeddings will override this via implementing the PromptMixin interface.

@property
has_embeddings = (source) ¶

overridden in fiftyone.utils.eta.ETAModel

Whether this instance can generate embeddings.

This method returns False by default. Methods that can generate embeddings will override this via implementing the EmbeddingsMixin interface.

@property
has_logits = (source) ¶

overridden in fiftyone.utils.eta.ETAModel, fiftyone.utils.torch.TorchImageModel

Whether this instance can generate logits for its predictions.

This method returns False by default. Methods that can generate logits will override this via implementing the LogitsMixin interface.

@property
media_type = (source) ¶

The media type processed by the model.

Supported values are "image" and "video".

@property
preprocess = (source) ¶

overridden in fiftyone.utils.eta.ETAModel, fiftyone.utils.torch.TorchImageModel, fiftyone.utils.transformers.FiftyOneTransformer, fiftyone.utils.transformers.FiftyOneZeroShotTransformer, fiftyone.utils.ultralytics.FiftyOneRTDETRModel, fiftyone.utils.ultralytics.FiftyOneYOLOModel

Whether to apply transforms during inference (True) or to assume that they have already been applied (False).

@property
ragged_batches = (source) ¶

True/False whether transforms may return tensors of different sizes. If True, then passing ragged lists of data to predict_all is not allowed.

@property
transforms = (source) ¶

The preprocessing function that will/must be applied to each input before prediction, or None if no preprocessing is performed.