fiftyone.utils.clip.zoo.TorchCLIPModel

class documentation

class TorchCLIPModel(fout.TorchImageModel, fom.PromptMixin): (source)

Constructor: TorchCLIPModel(config)

Torch implementation of CLIP from https://github.com/openai/CLIP.

Parameters
config	a `TorchCLIPModelConfig`

Method	`__init__`	Undocumented
Method	`embed_prompt`	Generates an embedding for the given text prompt.
Method	`embed_prompts`	Generates an embedding for the given text prompts.
Property	`can_embed_prompts`	Whether this instance can generate prompt embeddings.
Method	`_download_model`	Undocumented
Method	`_embed_prompts`	Undocumented
Method	`_get_class_logits`	Undocumented
Method	`_get_text_features`	Undocumented
Method	`_load_model`	Undocumented
Method	`_predict_all`	Applies a forward pass to the given iterable of data and returns the raw model output with no processing applied.
Instance Variable	`_text_features`	Undocumented
Instance Variable	`_tokenizer`	Undocumented

Inherited from TorchImageModel:

Method	`__enter__`	Undocumented
Method	`__exit__`	Undocumented
Method	`predict`	Performs prediction on the given image.
Method	`predict_all`	Performs prediction on the given batch of images.
Method	`preprocess.setter`	Undocumented
Instance Variable	`config`	Undocumented
Property	`classes`	The list of class labels for the model, if known.
Property	`device`	The `torch:torch.torch.device` that the model is using.
Property	`has_logits`	Whether this instance can generate logits.
Property	`mask_targets`	The mask targets for the model, if any.
Property	`media_type`	The media type processed by the model.
Property	`num_classes`	The number of classes for the model, if known.
Property	`preprocess`	Whether to apply preprocessing transforms for inference, if any.
Property	`ragged_batches`	Whether `transforms` may return tensors of different sizes. If True, then passing ragged lists of images to `predict_all` may not be not allowed.
Property	`skeleton`	The keypoint skeleton for the model, if any.
Property	`transforms`	A `torchvision.transforms` function that will be applied to each input before prediction, if any.
Property	`using_gpu`	Whether the model is using GPU.
Property	`using_half_precision`	Whether the model is using half precision.
Method	`_build_output_processor`	Undocumented
Method	`_build_transforms`	Undocumented
Method	`_forward_pass`	Undocumented
Method	`_load_transforms`	Undocumented
Method	`_parse_classes`	Undocumented
Method	`_parse_mask_targets`	Undocumented
Method	`_parse_skeleton`	Undocumented
Instance Variable	`_benchmark_orig`	Undocumented
Instance Variable	`_classes`	Undocumented
Instance Variable	`_device`	Undocumented
Instance Variable	`_mask_targets`	Undocumented
Instance Variable	`_model`	Undocumented
Instance Variable	`_no_grad`	Undocumented
Instance Variable	`_output_processor`	Undocumented
Instance Variable	`_preprocess`	Undocumented
Instance Variable	`_ragged_batches`	Undocumented
Instance Variable	`_skeleton`	Undocumented
Instance Variable	`_transforms`	Undocumented
Instance Variable	`_using_gpu`	Undocumented
Instance Variable	`_using_half_precision`	Undocumented

Inherited from TorchEmbeddingsMixin (via TorchImageModel):

Method	`embed`	Generates an embedding for the given data.
Method	`embed_all`	Generates embeddings for the given iterable of data.
Method	`get_embeddings`	Returns the embeddings generated by the last forward pass of the model.
Property	`has_embeddings`	Whether this instance has embeddings.
Instance Variable	`_as_feature_extractor`	Undocumented
Instance Variable	`_embeddings_layer`	Undocumented

Inherited from LogitsMixin (via TorchImageModel, TorchEmbeddingsMixin, EmbeddingsMixin, TorchModelMixin):

Method	`store_logits.setter`	Undocumented
Property	`store_logits`	Whether the model should store logits in its predictions.
Instance Variable	`_store_logits`	Undocumented

def __init__(self, config): (source) ¶

overrides fiftyone.utils.torch.TorchImageModel.__init__

Undocumented

def embed_prompt(self, prompt): (source) ¶

overrides fiftyone.core.models.PromptMixin.embed_prompt

Generates an embedding for the given text prompt.

Parameters
prompt	a text string
Returns
a numpy vector

def embed_prompts(self, prompts): (source) ¶

overrides fiftyone.core.models.PromptMixin.embed_prompts

Generates an embedding for the given text prompts.

Parameters
prompts	an iterable of text strings
Returns
a `num_prompts x num_dims` array of prompt embeddings

@property
can_embed_prompts = (source) ¶

overrides fiftyone.core.models.PromptMixin.can_embed_prompts

Whether this instance can generate prompt embeddings.

def _download_model(self, config): (source) ¶

overrides fiftyone.utils.torch.TorchImageModel._download_model

Undocumented

def _embed_prompts(self, prompts): (source) ¶

Undocumented

def _get_class_logits(self, text_features, image_features): (source) ¶

Undocumented

def _get_text_features(self): (source) ¶

Undocumented

def _load_model(self, config): (source) ¶

overrides fiftyone.utils.torch.TorchImageModel._load_model

Undocumented

def _predict_all(self, imgs): (source) ¶

overrides fiftyone.utils.torch.TorchImageModel._predict_all

Applies a forward pass to the given iterable of data and returns the raw model output with no processing applied.

Parameters
imgs	Undocumented
args	an iterable of data. See `predict_all` for details
Returns
the raw output of the model

_text_features = (source) ¶

Undocumented

_tokenizer = (source) ¶

Undocumented