fiftyone.utils.transformers.FiftyOneZeroShotTransformerForImageClassification

class documentation

class FiftyOneZeroShotTransformerForImageClassification(FiftyOneZeroShotTransformer): (source)

FiftyOne wrapper around a transformers model for zero-shot image classification.

Parameters
config	a `FiftyOneZeroShotTransformerConfig`

Method	`predict`	Performs prediction on the given data.
Method	`predict_all`	Performs prediction on the given iterable of data.
Method	`_load_model`	Undocumented
Method	`_predict_all_from_features`	Undocumented
Method	`_predict_all_from_retrieval`	Undocumented
Method	`_predict_from_retrieval`	Undocumented

Method	`__init__`	Undocumented
Instance Variable	`classes`	Undocumented
Instance Variable	`config`	Undocumented
Instance Variable	`device`	Undocumented
Instance Variable	`model`	Undocumented
Instance Variable	`processor`	Undocumented
Property	`media_type`	The media type processed by the model.
Property	`preprocess`	Whether to apply `transforms` during inference (True) or to assume that they have already been applied (False).
Property	`ragged_batches`	True/False whether `transforms` may return tensors of different sizes. If True, then passing ragged lists of data to `predict_all` is not allowed.
Property	`transforms`	The preprocessing function that will/must be applied to each input before prediction, or `None` if no preprocessing is performed.
Method	`_get_text_prompts`	Undocumented
Method	`_load_processor`	Undocumented
Instance Variable	`_text_prompts`	Undocumented

Method	`embed`	Generates an embedding for the given data.
Method	`embed_all`	Generates embeddings for the given iterable of data.
Property	`has_embeddings`	Whether this instance has embeddings.
Method	`_embed`	Undocumented

Method get_embeddings Returns the embeddings generated by the last forward pass of the model.

Method	`embed_prompt`	Generates an embedding for the given text prompt.
Method	`embed_prompts`	Generates an embedding for the given text prompts.
Property	`can_embed_prompts`	Whether this instance can generate prompt embeddings.
Method	`_embed_prompts`	Undocumented

Method	`__enter__`	Undocumented
Method	`__exit__`	Undocumented
Method	`preprocess.setter`	Undocumented
Property	`has_logits`	Whether this instance can generate logits for its predictions.

overrides fiftyone.utils.transformers.FiftyOneZeroShotTransformer.predict

Performs prediction on the given data.

Image models should support, at minimum, processing arg values that are uint8 numpy arrays (HWC).

Video models should support, at minimum, processing arg values that are eta.core.video.VideoReader instances.

Parameters
arg	the data
Returns
a `fiftyone.core.labels.Label` instance or dict of `fiftyone.core.labels.Label` instances containing the predictions

def predict_all(self, args): (source) ¶

overrides fiftyone.core.models.Model.predict_all

Performs prediction on the given iterable of data.

Image models should support, at minimum, processing args values that are either lists of uint8 numpy arrays (HWC) or numpy array tensors (NHWC).

Video models should support, at minimum, processing args values that are lists of eta.core.video.VideoReader instances.

Subclasses can override this method to increase efficiency, but, by default, this method simply iterates over the data and applies predict to each.

Parameters
args	an iterable of data
Returns
a list of `fiftyone.core.labels.Label` instances or a list of dicts of `fiftyone.core.labels.Label` instances containing the predictions

def _load_model(self, config): (source) ¶

overrides fiftyone.utils.transformers.FiftyOneZeroShotTransformer._load_model

Undocumented

def _predict_all_from_features(self, args): (source) ¶

Undocumented

def _predict_all_from_retrieval(self, args): (source) ¶

Undocumented

def _predict_from_retrieval(self, arg): (source) ¶

Undocumented