Model Zoo API Reference ¶¶

You can interact with the Model Zoo either via the Python library or the CLI.

Listing zoo models ¶¶

Getting information about zoo models ¶¶

Downloading zoo models ¶¶

Installing zoo model requirements ¶¶

Loading zoo models ¶¶

You can load a zoo model via load_zoo_model().

By default, the model will be automatically downloaded from the web the first time you access it if it is not already downloaded:

import fiftyone.zoo as foz

# The model will be downloaded from the web the first time you access it
model = foz.load_zoo_model("faster-rcnn-resnet50-fpn-coco-torch")

You can also provide additional arguments to load_zoo_model() to customize the import behavior:

# Load the zoo model and install any necessary requirements in order to
# use it (logging warnings if any issues arise)
model = foz.load_zoo_model(
    "faster-rcnn-resnet50-fpn-coco-torch",
    install_requirements=True,
    error_level=1,
)

Note

By default, FiftyOne will attempt to ensure that any requirements such as Python packages or CUDA versions are satisfied before loading the model, and an error will be raised if a requirement is not satisfied.

You can customize this behavior via the error_level argument to load_zoo_model(), or you can permanently adjust this behavior by setting the requirement_error_level parameter of your FiftyOne config.

An error_level of 0 will raise an error if a requirement is not satisfied, 1 will log a warning if the requirement is not satisfied, and 2 will ignore unsatisfied requirements.

If you are using a conda environment, it is recommended you use an error_level of 1 or 2, since FiftyOne uses pip to check for requirements.

Applying zoo models ¶¶

Generating embeddings with zoo models ¶¶

Controlling where zoo models are downloaded ¶¶

By default, zoo models are downloaded into subdirectories of fiftyone.config.model_zoo_dir corresponding to their names.

You can customize this backend by modifying the model_zoo_dir setting of your FiftyOne config.

Deleting zoo models ¶¶

Adding models to the zoo ¶¶

We frequently add new models to the Model Zoo, which will automatically become accessible to you when you update your FiftyOne package.

Note

FiftyOne is open source! You are welcome to contribute models to the public model zoo by submitting a pull request to the GitHub repository.

You can also add your own models to your local model zoo, enabling you to work with these models via the fiftyone.zoo package and the CLI using the same syntax that you would with publicly available models.

To add model(s) to your local zoo, you simply write a JSON manifest file in the format below to tell FiftyOne about the model(s). For example, the manifest below adds a second copy of the yolo-v2-coco-tf1 model to the zoo under the alias yolo-v2-coco-tf1-high-conf that only returns predictions whose confidence is at least 0.5:

{
    "models": [\
        {\
            "base_name": "yolo-v2-coco-tf1-high-conf",\
            "base_filename": "yolo-v2-coco-high-conf.weights",\
            "version": null,\
            "description": "A YOLOv2 model with confidence threshold set to 0.5",\
            "manager": {\
                "type": "fiftyone.core.models.ModelManager",\
                "config": {\
                    "google_drive_id": "1ajuPZws47SOw3xJc4Wvk1yuiB3qv8ycr"\
                }\
            },\
            "default_deployment_config_dict": {\
                "type": "fiftyone.utils.eta.ETAModel",\
                "config": {\
                    "type": "eta.detectors.YOLODetector",\
                    "config": {\
                        "config_dir": "{{eta}}/tensorflow/darkflow/cfg/",\
                        "config_path": "{{eta}}/tensorflow/darkflow/cfg/yolo.cfg",\
                        "confidence_thresh": 0.5\
                    }\
                }\
            },\
            "requirements": {\
                "cpu": {\
                    "support": true,\
                    "packages": ["tensorflow<2"]\
                },\
                "gpu": {\
                    "support": true,\
                    "packages": ["tensorflow-gpu<2"]\
                }\
            },\
            "tags": ["detection", "coco", "tf1"],\
            "date_added": "2020-12-11 13:45:51"\
        }\
    ]
}

Note

Adjusting the hard-coded threshold of the above model is possible via JSON-only changes in this case because the underlying eta.detectors.YOLODetector class exposes this as a parameter.

In practice, there is no need to hard-code confidence thresholds in models, since the apply_model() method supports supplying an optional confidence threshold that is applied post-facto to the predictions generated by any model.

Models manifest JSON files should have a models key that contains a list of serialized ZooModel class definitions that describe how to download and load the model.

Finally, expose your new models(s) to FiftyOne by adding your manifest to the model_zoo_manifest_paths parameter of your FiftyOne config. One way to do this is to set the FIFTYONE_MODEL_ZOO_MANIFEST_PATHS environment variable:

export FIFTYONE_MODEL_ZOO_MANIFEST_PATHS=/path/to/custom/manifest.json

Now you can load and apply the yolo-v2-coco-tf1-high-conf model as you would any other zoo model:

import fiftyone as fo
import fiftyone.zoo as foz

# Load custom model
model = foz.load_zoo_model("yolo-v2-coco-tf1-high-conf")

# Apply model to a dataset
dataset = fo.load_dataset(...)
dataset.apply_model(model, label_field="predictions")