FiftyOne Teams Installation ¶¶

FiftyOne Teams deployments come with a centralized FiftyOne Teams App and database that allows your entire team to collaborate securely on the same datasets. FiftyOne Teams is deployed entirely into your environment, either on-premises or in a private cloud. Your data never leaves your environment.

FiftyOne Teams can be deployed on a wide variety of infrastructure solutions, including Kubernetes and Docker.

Note

Detailed instructions for the initial FiftyOne Teams deployment, along with all necessary components, are made available by your Voxel51 CS engineer during the onboarding process.

Python SDK ¶¶

While the FiftyOne Teams App allows for countless new App-centric workflows, any existing Python-based workflows that you’ve fallen in love with in the open-source version of FiftyOne are still directly applicable!

FiftyOne Teams requires an updated Python SDK, which is a wrapper around the open-source FiftyOne package that adds new functionality like support for cloud-backed media.

You can find the installation instructions under the “Install FiftyOne” section of the Teams App by clicking on your user icon in the upper right corner:

There you’ll see instructions for installing a fiftyone package from the private PyPI server as shown below:

pip install --index-url https://${TOKEN}@pypi.fiftyone.ai fiftyone

Note

See Installation with Poetry if you use poetry instead of pip.

Note

The Teams Python package is named fiftyone and has the same module structure as fiftyone, so any existing scripts you built using open source will continue to run after you upgrade!

Next Steps ¶¶

After installing the Teams Python SDK in your virtual environment, you’ll need to configure two things:

Your team’s API connection or MongoDB connection
The cloud credentials to access your cloud-backed media

That’s it! Any operations you perform will be stored in a centralized location and will be available to all users with access to the same datasets in the Teams App or their Python workflows.

Installation with Poetry ¶¶

If you are using poetry to install your dependencies rather than pip, you will need to follow instructions in the docs for installing from a private repository. The two key points are specifying the additional private source and declaring that the fiftyone module should be found there and not the default PyPI location.

Add source ¶¶

In poetry v1.5, it is recommended to use an explicit package source.

poetry source add --priority=explicit fiftyone-teams https://pypi.fiftyone.ai/simple/

Prior to v1.5, you should use the deprecated secondary package source.

poetry source add --secondary fiftyone-teams https://pypi.fiftyone.ai/simple/

Configure credentials ¶¶

poetry config http-basic.fiftyone-teams ${TOKEN} ""

Alternatively, you can specify the credentials in environment variables.

export POETRY_HTTP_BASIC_FIFTYONE_TEAMS_USERNAME="${TOKEN}"
export POETRY_HTTP_BASIC_FIFTYONE_TEAMS_PASSWORD=""

If you have trouble configuring the credentials, see more in the poetry docs here.

Add fiftyone dependency ¶¶

Replace X.Y.Z with the proper version

poetry add --source fiftyone-teams fiftyone==X.Y.Z

Note

Due to an unresolved misalignment with poetry and a FiftyOne dependency, kaleido, you must add it to your own dependencies as well:

poetry add kaleido==0.2.1

You should then see snippets in the pyproject.toml file like the following (the priority line will be different for poetry<v1.5):

[[tool.poetry.source]]
name = "fiftyone-teams"
url = "https://pypi.fiftyone.ai"
priority = "explicit"

[tool.poetry.dependencies]
fiftyone = {version = "X.Y.Z", source = "fiftyone-teams}

Cloud credentials ¶¶

In order to utilize cloud-backed media functionality of FiftyOne Teams, at least one cloud source must be configured with proper credentials. Below are instructions for configuring each supported cloud provider for local SDK use or directly to the Teams containers. An admin can also configure\ credentials for use by all app users.

If your datasets include cloud-backed point clouds or segmentation maps, you may need to configure cross-origin resource sharing (CORS) for your cloud buckets. Details are provided below for each cloud platform.

Browser caching ¶¶

If your datasets include cloud-backed media, we strongly recommend configuring your data sources to allow for built in browser caching. This will cache signed URL responses so you don’t need to reload assets from your cloud storage between sessions. Details are provided below for each cloud platform.

Amazon S3 ¶¶

To work with FiftyOne datasets whose media are stored in Amazon S3, you simply need to provide AWS credentials to your Teams client with read access to the relevant objects and buckets.

You can do this in any of the following ways:

1. Configure/provide AWS credentials in any format supported by the boto3 library. For example, here are two of the supported methods:

# Access key
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # if applicable
export AWS_DEFAULT_REGION=...

# Web identity provider
export AWS_ROLE_ARN=...
export AWS_WEB_IDENTITY_TOKEN_FILE=...
export AWS_ROLE_SESSION_NAME... #if applicable
export AWS_DEFAULT_REGION=...

2. Provide AWS credentials on a per-session basis by setting one of the following sets of environment variables to point to your AWS credentials on disk:

# AWS config file
export AWS_CONFIG_FILE="/path/to/aws-config.ini"
export AWS_PROFILE=default  # optional

# Shared credentials file
export AWS_SHARED_CREDENTIALS_FILE="/path/to/aws-credentials.ini"
export AWS_PROFILE=default  # optional

In the above, the config file should use this syntax and the shared credentials file should use this syntax.

Note

FiftyOne Teams requires either the s3:ListBucket or s3:GetBucketLocation permission in order to access objects in S3 buckets.

If you wish to use multi-account credentials, your credentials must have the s3:ListBucket permission, as s3:GetBucketLocation does not support this.

3. Permanently register AWS credentials on a particular machine by adding the following keys to your media cache config:

{
    "aws_config_file": "/path/to/aws-config.ini",
    "aws_profile": "default"  # optional
}

If you need to configure CORS on your AWS buckets, here is an example configuration:

[\
    {\
        "origin": ["https://fiftyone-teams-deployment.yourcompany.com"],\
        "method": ["GET", "HEAD"],\
        "responseHeader": ["*"],\
        "maxAgeSeconds": 86400\
    }\
]

If you would like to take advantage of browser caching you can specify cache-control headers on S3 objects. By default S3 does not provide cache-control headers so it will be up to your browser’s heuristics engine to determine how long to cache the object.

Google Cloud Storage ¶¶

To work with FiftyOne datasets whose media are stored in Google Cloud Storage, you simply need to provide credentials to your Teams client with read access to the relevant objects and buckets.

You can do this in any of the following ways:

1. Configure application default credentials in a manner supported by Google Cloud, such as:

2. Provide GCS credentials on a per-session basis by setting the following environment variables to point to your GCS credentials on disk:

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/gcp-credentials.json"

3. Permanently register GCS credentials on a particular machine by adding the following keys to your media cache config:

{
    "google_application_credentials": "/path/to/gcp-credentials.json"
}

In the above, the credentials file can contain any format supported by google.auth.load_credentials_from_file(), which includes a service account key, stored authorized user credentials, or external account credentials.

If you need to configure CORS on your GCP buckets, here is an example configuration:

[\
    {\
        "origin": ["https://fiftyone-teams-deployment.yourcompany.com"],\
        "method": ["GET", "HEAD"],\
        "responseHeader": ["*"],\
        "maxAgeSeconds": 3600\
    }\
]

If you would like to take advantage of browser caching you can specify cache-control headers on GCP content. By default GCP sets the max-age=0 seconds meaning no caching will occur.

Microsoft Azure ¶¶

To work with FiftyOne datasets whose media are stored in Azure Storage, you simply need to provide Azure credentials to your Teams client with read access to the relevant objects and containers.

You can do this in any of the following ways:

1. Provide your Azure credentials in any manner recognized by azure.identity.DefaultAzureCredential

2. Provide your Azure credentials on a per-session basis by setting any group of environment variables shown below:

# Option 1
export AZURE_STORAGE_CONNECTION_STRING=...
export AZURE_ALIAS=...  # optional

# Option 2
export AZURE_STORAGE_ACCOUNT=...
export AZURE_STORAGE_KEY=...
export AZURE_ALIAS=...  # optional

# Option 3
export AZURE_STORAGE_ACCOUNT=...
export AZURE_CLIENT_ID=...
export AZURE_CLIENT_SECRET=...
export AZURE_TENANT_ID=...
export AZURE_ALIAS=...  # optional

3. Provide Azure credentials on a per-session basis by setting the following environment variables to point to your Azure credentials on disk:

export AZURE_CREDENTIALS_FILE=/path/to/azure-credentials.ini
export AZURE_PROFILE=default  # optional

4. Permanently register Azure credentials on a particular machine by adding the following keys to your media cache config:

{
    "azure_credentials_file": "/path/to/azure-credentials.ini",
    "azure_profile": "default"  # optional
}

In the options above, the .ini file should have syntax similar to one of the following:

[default]
conn_str = ...
alias = ...  # optional

[default]
account_name = ...
account_key = ...
alias = ...  # optional

[default]
account_name = ...
client_id = ...
secret = ...
tenant = ...
alias = ...  # optional

When populating samples with Azure Storage filepaths, you can either specify paths by their full URL:

filepath = "https://${account_name}.blob.core.windows.net/container/path/to/object.ext"

# For example
filepath = "https://voxel51.blob.core.windows.net/test-container/image.jpg"

or, if you have defined an alias in your config, you may instead prefix the alias:

filepath = "${alias}://container/path/to/object.ext"

# For example
filepath = "az://test-container/image.jpg"

Note

If you use a custom Azure domain, you can provide it by setting the AZURE_STORAGE_ACCOUNT_URL environment variable or by including the account_url key in your credentials .ini file.

If you would like to take advantage of browser caching you can specify cache-control headers on Azure blobs. By default Azure does not provide cache-control headers so it will be up to your browser’s heuristics engine to determine how long to cache the object.

MinIO ¶¶

To work with FiftyOne datasets whose media are stored in MinIO, you simply need to provide the credentials to your Teams client with read access to the relevant objects and buckets.

You can do this in any of the following ways:

1. Provide your MinIO credentials on a per-session basis by setting the individual environment variables shown below:

export MINIO_ACCESS_KEY=...
export MINIO_SECRET_ACCESS_KEY=...
export MINIO_ENDPOINT_URL=...
export MINIO_ALIAS=...  # optional
export MINIO_REGION=...  # if applicable

2. Provide MinIO credentials on a per-session basis by setting the following environment variables to point to your MinIO credentials on disk:

export MINIO_CONFIG_FILE=/path/to/minio-config.ini
export MINIO_PROFILE=default  # optional

3. Permanently register MinIO credentials on a particular machine by adding the following keys to your media cache config:

{
    "minio_config_file": "/path/to/minio-config.ini",
    "minio_profile": "default"  # optional
}

In the options above, the .ini file should have syntax similar the following:

[default]
access_key = ...
secret_access_key = ...
endpoint_url = ...
alias = ...  # optional
region = ...  # if applicable

When populating samples with MinIO filepaths, you can either specify paths by prefixing your MinIO endpoint URL:

filepath = "${endpoint_url}/bucket/path/to/object.ext"

# For example
filepath = "https://voxel51.min.io/test-bucket/image.jpg"

or, if you have defined an alias in your config, you may instead prefix the alias:

filepath = "${alias}://bucket/path/to/object.ext"

# For example
filepath = "minio://test-bucket/image.jpg"

If you would like to take advantage of browser caching you can specify cache-control headers on MinIO content using the metadata field of the put_object API. By default Minio does not provide cache-control headers so it will be up to your browser’s heuristics engine to determine how long to cache the object.

Extra client arguments ¶¶

Configuring credentials following the instructions above is almost always sufficient for FiftyOne Teams to properly utilize them. In rare cases where the cloud provider client needs non-default configuration, you can add extra client kwargs via the media cache config:

{
    "extra_client_kwargs": {
        "azure": {"extra_kwarg": "value"},
        "gcs": {"extra_kwarg": "value"},
        "minio": {"extra_kwarg": "value"},
        "s3": {"extra_kwarg": "value"}
    }
}

Provider names and the class that extra kwargs are passed to:

azure: azure.identity.DefaultAzureCredential
gcs: google.cloud.storage.Client
minio: botocore.config.Config
s3: botocore.config.Config

Cloud storage page ¶¶

Admins can also configure cloud credentials via the Settings > Cloud storage page.

Credentials configured via this page are stored (encrypted) in the Teams database, rather than needing to be configured through environment variables in your Teams deployment.

Note

Any credentials configured via environment variables in your deployment will not be displayed in this page.

To upload a new credential, click the Add credential button:

cloud-creds-add-credentials-button

This will open a modal that you can use to add a credential for any of the available providers:

blank-cloud-creds-modal

After the appropriate files or fields are populated, click Save credential to store the (encrypted) credential.

As depicted in the screenshot above, a credential can optionally be restricted to a specific list of bucket(s):

If one or more buckets are provided, the credentials are bucket-specific credentials that will only be used to read/write media within the specified bucket(s)
If no buckets are provided, the credentials are default credentials that will be used whenever trying to read/write any media for the provider that does not belong to a bucket with bucket-specific credentials

Note

Bucket-specific credentials are useful in situations where you cannot or do not wish to provide a single set of credentials to cover all buckets that your team plans to use within a given cloud storage provider.

When providing bucket-specific credentials, you may either provide bucket names like my-bucket, or you can provide fully-qualified buckets like s3://my-bucket and https://voxel51.blob.core.windows.net/my-container.

Alternatively, credentials can be updated programmatically with the add_cloud_credentials() method in the Management SDK.

Any cloud credentials uploaded via this method will automatically be used by the Teams UI when any user attempts to load media associated with the appropriate provider or specific bucket.

Note

By default, Teams servers refresh their credentials every 120 seconds, so you may need to wait up to two minutes after modifying your credentials via this page in order for the changes to take effect.

Note

Users cannot access stored credentials directly, either via the Teams UI or by using the Teams SDK locally. The credentials are only decrypted and used internally by the Teams servers.

FiftyOne Teams Installation ¶¶

Python SDK ¶¶

Next Steps ¶¶

Installation with Poetry ¶¶

Add source ¶¶

Configure credentials ¶¶

Add fiftyone dependency ¶¶

Cloud credentials ¶¶

Cross-origin resource sharing (CORS) ¶¶

Browser caching ¶¶

Amazon S3 ¶¶

Google Cloud Storage ¶¶

Microsoft Azure ¶¶

MinIO ¶¶

Extra client arguments ¶¶

Cloud storage page ¶¶