fiftyone.utils.transformers

module documentation

(source)

Utilities for working with Hugging Face Transformers.

Class	`FiftyOneTransformer`	FiftyOne wrapper around a `transformers` model.
Class	`FiftyOneTransformerConfig`	Configuration for a `FiftyOneTransformer`.
Class	`FiftyOneTransformerForDepthEstimation`	FiftyOne wrapper around a `transformers` model for depth estimation.
Class	`FiftyOneTransformerForDepthEstimationConfig`	Configuration for a `FiftyOneTransformerForDepthEstimation`.
Class	`FiftyOneTransformerForImageClassification`	FiftyOne wrapper around a `transformers` model for image classification.
Class	`FiftyOneTransformerForImageClassificationConfig`	Configuration for a `FiftyOneTransformerForImageClassification`.
Class	`FiftyOneTransformerForObjectDetection`	FiftyOne wrapper around a `transformers` model for object detection.
Class	`FiftyOneTransformerForObjectDetectionConfig`	Configuration for a `FiftyOneTransformerForObjectDetection`.
Class	`FiftyOneTransformerForSemanticSegmentation`	FiftyOne wrapper around a `transformers` model for semantic segmentation.
Class	`FiftyOneTransformerForSemanticSegmentationConfig`	Configuration for a `FiftyOneTransformerForSemanticSegmentation`.
Class	`FiftyOneZeroShotTransformer`	FiftyOne wrapper around a `transformers` model.
Class	`FiftyOneZeroShotTransformerConfig`	Configuration for a `FiftyOneZeroShotTransformer`.
Class	`FiftyOneZeroShotTransformerForImageClassification`	FiftyOne wrapper around a `transformers` model for zero-shot image classification.
Class	`FiftyOneZeroShotTransformerForImageClassificationConfig`	Configuration for a `FiftyOneZeroShotTransformerForImageClassification`.
Class	`FiftyOneZeroShotTransformerForObjectDetection`	FiftyOne wrapper around a `transformers` model for zero shot object detection.
Class	`FiftyOneZeroShotTransformerForObjectDetectionConfig`	Configuration for a `FiftyOneZeroShotTransformerForObjectDetection`.
Class	`TransformerEmbeddingsMixin`	Mixin for Transformers that can generate embeddings.
Class	`ZeroShotTransformerEmbeddingsMixin`	Mixin for Transformers that can generate embeddings.
Class	`ZeroShotTransformerPromptMixin`	Mixin for Transformers that can perform zero-shot prediction.
Function	`convert_transformers_model`	Converts the given Hugging Face transformers model into a FiftyOne model.
Function	`get_model_type`	Returns the string model type for the given model.
Function	`to_classification`	Converts the Transformers classification results to FiftyOne format.
Function	`to_detections`	Converts the Transformers detection results to FiftyOne format.
Function	`to_heatmap`	Converts the Transformers depth estimation results to FiftyOne format.
Function	`to_segmentation`	Converts the Transformers semantic segmentation results to FiftyOne format.
Constant	`DEFAULT_CLASSIFICATION_PATH`	Undocumented
Constant	`DEFAULT_DEPTH_ESTIMATION_PATH`	Undocumented
Constant	`DEFAULT_DETECTION_PATH`	Undocumented
Constant	`DEFAULT_SEGMENTATION_PATH`	Undocumented
Constant	`DEFAULT_ZERO_SHOT_CLASSIFICATION_PATH`	Undocumented
Constant	`DEFAULT_ZERO_SHOT_DETECTION_PATH`	Undocumented
Variable	`logger`	Undocumented
Variable	`torch`	Undocumented
Variable	`transformers`	Undocumented
Function	`_convert_bounding_box`	Undocumented
Function	`_convert_transformer_base_model`	Undocumented
Function	`_convert_transformer_for_depth_estimation`	Undocumented
Function	`_convert_transformer_for_image_classification`	Undocumented
Function	`_convert_transformer_for_object_detection`	Undocumented
Function	`_convert_transformer_for_semantic_segmentation`	Undocumented
Function	`_convert_zero_shot_transformer_for_image_classification`	Undocumented
Function	`_convert_zero_shot_transformer_for_object_detection`	Undocumented
Function	`_create_heatmap`	Undocumented
Function	`_create_segmentation`	Undocumented
Function	`_get_base_model_name`	Undocumented
Function	`_get_class`	Undocumented
Function	`_get_detector_from_processor`	Undocumented
Function	`_get_image_processor`	Undocumented
Function	`_get_image_processor_fallback`	Undocumented
Function	`_get_model_for_image_text_retrieval`	Undocumented
Function	`_get_model_type_string`	Undocumented
Function	`_get_processor`	Undocumented
Function	`_has_detection_model`	Undocumented
Function	`_has_image_text_retrieval`	Undocumented
Function	`_has_text_and_image_features`	Undocumented
Function	`_is_transformer_base_model`	Undocumented
Function	`_is_transformer_for_depth_estimation`	Undocumented
Function	`_is_transformer_for_image_classification`	Undocumented
Function	`_is_transformer_for_object_detection`	Undocumented
Function	`_is_transformer_for_semantic_segmentation`	Undocumented
Function	`_is_zero_shot_model`	Undocumented
Function	`_to_detections`	Undocumented

def convert_transformers_model(model, task=None): (source) ¶

Converts the given Hugging Face transformers model into a FiftyOne model.

Parameters
model	a `transformers` model
task:`None`	the task of the model. Supported values are `"image-classification"`, `"object-detection"`, `"semantic-segmentation"`, and `"depth-estimation"`. If not specified, the task is automatically inferred from the model
Returns
a `fiftyone.core.models.Model`
Raises
`ValueError`	if the model could not be converted

def get_model_type(model, task=None): (source) ¶

Returns the string model type for the given model.

If the model is a zero-shot model, the task is appended to the model type.

Parameters
model	a `transformers.model` model
task:`None`	an optional task type
Returns
the model type string

def to_classification(results, id2label): (source) ¶

Converts the Transformers classification results to FiftyOne format.

Parameters
results	Transformers classification results
id2label	Transformers ID to label mapping
Returns
a single or list of `fiftyone.core.labels.Classification`

def to_detections(results, id2label, image_sizes): (source) ¶

Converts the Transformers detection results to FiftyOne format.

Parameters
results	Transformers detection results
id2label	Transformers ID to label mapping
image_sizes	the list of image sizes
Returns
a single or list of `fiftyone.core.labels.Detections`

def to_heatmap(results): (source) ¶

Converts the Transformers depth estimation results to FiftyOne format.

Parameters
results	Transformers depth estimation results
Returns
a single or list of `fiftyone.core.labels.Heatmap`

def to_segmentation(results): (source) ¶

Converts the Transformers semantic segmentation results to FiftyOne format.

Parameters
results	Transformers semantic segmentation results
Returns
a single or list of `fiftyone.core.labels.Segmentation`

DEFAULT_CLASSIFICATION_PATH: str = (source) ¶

Undocumented

Value

'google/vit-base-patch16-224'

DEFAULT_DEPTH_ESTIMATION_PATH: str = (source) ¶

Undocumented

Value

'Intel/dpt-hybrid-midas'

DEFAULT_DETECTION_PATH: str = (source) ¶

Undocumented

Value

'hustvl/yolos-tiny'

DEFAULT_SEGMENTATION_PATH: str = (source) ¶

Undocumented

Value

'nvidia/segformer-b0-finetuned-ade-512-512'

DEFAULT_ZERO_SHOT_CLASSIFICATION_PATH: str = (source) ¶

Undocumented

Value

'openai/clip-vit-large-patch14'

DEFAULT_ZERO_SHOT_DETECTION_PATH: str = (source) ¶

Undocumented

Value

'google/owlvit-base-patch32'

logger = (source) ¶

Undocumented

torch = (source) ¶

Undocumented

transformers = (source) ¶

Undocumented

def _convert_bounding_box(box, image_shape): (source) ¶

Undocumented

def _convert_transformer_base_model(model): (source) ¶

Undocumented

def _convert_transformer_for_depth_estimation(model): (source) ¶

Undocumented

def _convert_transformer_for_image_classification(model): (source) ¶

Undocumented

def _convert_transformer_for_object_detection(model): (source) ¶

Undocumented

def _convert_transformer_for_semantic_segmentation(model): (source) ¶

Undocumented

def _convert_zero_shot_transformer_for_image_classification(model): (source) ¶

Undocumented

def _convert_zero_shot_transformer_for_object_detection(model): (source) ¶

Undocumented

def _create_heatmap(heatmap): (source) ¶

Undocumented

def _create_segmentation(mask): (source) ¶

Undocumented

def _get_base_model_name(model): (source) ¶

Undocumented

def _get_class(label, id2label): (source) ¶

Undocumented

def _get_detector_from_processor(processor, model_name_or_path): (source) ¶

Undocumented

def _get_image_processor(model): (source) ¶

Undocumented

def _get_image_processor_fallback(model): (source) ¶

Undocumented

def _get_model_for_image_text_retrieval(base_model, model_name_or_path): (source) ¶

Undocumented

def _get_model_type_string(model): (source) ¶

Undocumented

def _get_processor(model): (source) ¶

Undocumented

def _has_detection_model(model): (source) ¶

Undocumented

def _has_image_text_retrieval(model): (source) ¶

Undocumented

def _has_text_and_image_features(model): (source) ¶

Undocumented

def _is_transformer_base_model(model): (source) ¶

Undocumented

def _is_transformer_for_depth_estimation(model): (source) ¶

Undocumented

def _is_transformer_for_image_classification(model): (source) ¶

Undocumented

def _is_transformer_for_object_detection(model): (source) ¶

Undocumented

def _is_transformer_for_semantic_segmentation(model): (source) ¶

Undocumented

def _is_zero_shot_model(model): (source) ¶

Undocumented

def _to_detections(result, id2label, image_size): (source) ¶

Undocumented