fiftyone.core.dataset

module documentation

(source)

FiftyOne datasets.

Class	`Dataset`	A FiftyOne dataset.
Exception	`DatasetNotFoundError`	Exception raised when a dataset is not found.
Function	`dataset_exists`	Checks if the dataset exists.
Function	`delete_dataset`	Deletes the FiftyOne dataset with the given name.
Function	`delete_datasets`	Deletes all FiftyOne datasets whose names match the given glob pattern.
Function	`delete_non_persistent_datasets`	Deletes all non-persistent datasets.
Function	`get_default_dataset_dir`	Returns the default dataset directory for the dataset with the given name.
Function	`get_default_dataset_name`	Returns a default dataset name based on the current time.
Function	`list_datasets`	Lists the available FiftyOne datasets.
Function	`load_dataset`	Loads the FiftyOne dataset with the given name.
Function	`make_unique_dataset_name`	Makes a unique dataset name with the given root name.
Variable	`foos`	Undocumented
Variable	`fot`	Undocumented
Variable	`foud`	Undocumented
Variable	`logger`	Undocumented
Function	`_add_collection_with_new_ids`	Undocumented
Function	`_always_select_field`	Undocumented
Function	`_cleanup_frame_index`	Undocumented
Function	`_cleanup_index`	Undocumented
Function	`_clone_collection`	Undocumented
Function	`_clone_collection_indexes`	Undocumented
Function	`_clone_extras`	Undocumented
Function	`_clone_indexes`	Undocumented
Function	`_clone_reference_doc`	Undocumented
Function	`_clone_run`	Undocumented
Function	`_create_dataset`	Undocumented
Function	`_create_frame_document_cls`	Undocumented
Function	`_create_group_indexes`	Undocumented
Function	`_create_indexes`	Undocumented
Function	`_create_sample_document_cls`	Undocumented
Function	`_declare_fields`	Undocumented
Function	`_delete_dataset_doc`	Undocumented
Function	`_delete_non_persistent_datasets`	Undocumented
Function	`_discard_none`	Undocumented
Function	`_do_load_dataset`	Undocumented
Function	`_ensure_index`	Undocumented
Function	`_extract_archive_if_necessary`	Undocumented
Function	`_finalize_frames`	Undocumented
Function	`_get_collstats`	Undocumented
Function	`_get_frame_ids`	Undocumented
Function	`_get_frame_ids_for_sample`	Undocumented
Function	`_get_frames_pipeline`	Undocumented
Function	`_get_group_field`	Undocumented
Function	`_get_group_id`	Undocumented
Function	`_get_group_ids`	Undocumented
Function	`_get_indexes_to_skip`	Undocumented
Function	`_get_media_type`	Undocumented
Function	`_get_random_characters`	Undocumented
Function	`_get_sample_ids`	Undocumented
Function	`_get_samples_pipeline`	Undocumented
Function	`_get_single_index_map`	Undocumented
Function	`_handle_nested_fields`	Undocumented
Function	`_index_frames`	Undocumented
Function	`_list_datasets`	Undocumented
Function	`_list_datasets_info`	Undocumented
Function	`_list_datasets_query`	Undocumented
Function	`_load_clips_source_dataset`	Undocumented
Function	`_load_dataset`	Undocumented
Function	`_make_frame_collection_name`	Undocumented
Function	`_make_merge_samples_generator`	Undocumented
Function	`_make_sample_collection_name`	Undocumented
Function	`_merge_dataset_doc`	Undocumented
Function	`_merge_docs`	Undocumented
Function	`_merge_label_list_field`	Undocumented
Function	`_merge_list_field`	Undocumented
Function	`_merge_samples_pipeline`	Undocumented
Function	`_merge_samples_python`	Undocumented
Function	`_parse_field_mapping`	Undocumented
Function	`_parse_fields`	Undocumented
Function	`_save_view`	Undocumented
Function	`_set_field_read_only`	Undocumented
Function	`_to_list`	Undocumented
Function	`_update_no_overwrite`	Undocumented
Function	`_validate_dataset_name`	Validates that the given dataset name is available.
Constant	`_SUMMARY_FIELD_KEY`	Undocumented

def dataset_exists(name): (source) ¶

Checks if the dataset exists.

Parameters
name	the name of the dataset
Returns
True/False

def delete_dataset(name, verbose=False): (source) ¶

Deletes the FiftyOne dataset with the given name.

Parameters
name	the name of the dataset
verbose:`False`	whether to log the name of the deleted dataset

def delete_datasets(glob_patt, verbose=False): (source) ¶

Deletes all FiftyOne datasets whose names match the given glob pattern.

Parameters
glob_patt	a glob pattern of datasets to delete
verbose:`False`	whether to log the names of deleted datasets

def delete_non_persistent_datasets(verbose=False): (source) ¶

Deletes all non-persistent datasets.

Parameters
verbose:`False`	whether to log the names of deleted datasets

def get_default_dataset_dir(name): (source) ¶

Returns the default dataset directory for the dataset with the given name.

Parameters
name	the dataset name
Returns
the default directory for the dataset

def get_default_dataset_name(): (source) ¶

Returns a default dataset name based on the current time.

Returns
a dataset name

def list_datasets(glob_patt=None, tags=None, info=False): (source) ¶

Lists the available FiftyOne datasets.

Parameters
glob_patt:`None`	an optional glob pattern of names to return
tags:`None`	only include datasets that have the specified tag or list of tags
info:`False`	whether to return info dicts describing each dataset rather than just their names
Returns
a list of dataset names or info dicts

def load_dataset(name, create_if_necessary=False): (source) ¶

Loads the FiftyOne dataset with the given name.

To create a new dataset, use the Dataset constructor.

Note

Dataset instances are singletons keyed by their name, so all calls to this method with a given dataset name in a program will return the same object.

Parameters
name	the name of the dataset
create_if_necessary:`False`	if no dataset exists, create an empty one
Returns
a `Dataset`
Raises
`DatasetNotFoundError`	if the dataset does not exist and `create_if_necessary` is False

def make_unique_dataset_name(root): (source) ¶

Makes a unique dataset name with the given root name.

Parameters
root	the root name for the dataset
Returns
the dataset name

foos = (source) ¶

Undocumented

fot = (source) ¶

Undocumented

foud = (source) ¶

Undocumented

logger = (source) ¶

Undocumented

def _add_collection_with_new_ids(dataset, sample_collection, include_info=True, overwrite_info=False): (source) ¶

Undocumented

def _always_select_field(sample_collection, field): (source) ¶

Undocumented

def _cleanup_frame_index(dataset, index): (source) ¶

Undocumented

def _cleanup_index(dataset, db_field, new_index, dropped_index): (source) ¶

Undocumented

def _clone_collection(sample_collection, name, persistent): (source) ¶

Undocumented

def _clone_collection_indexes(src_collection_name, dst_collection_name, skip=None): (source) ¶

Undocumented

def _clone_extras(src_dataset, dst_dataset, now): (source) ¶

Undocumented

def _clone_indexes(src_collection, dst_doc): (source) ¶

Undocumented

def _clone_reference_doc(ref_doc): (source) ¶

Undocumented

def _clone_run(run_doc): (source) ¶

Undocumented

def _create_dataset(obj, name, persistent=False, _patches=False, _frames=False, _clips=False, _src_collection=None): (source) ¶

Undocumented

def _create_frame_document_cls(dataset, frame_collection_name, field_docs=None): (source) ¶

Undocumented

def _create_group_indexes(sample_collection_name, group_field): (source) ¶

Undocumented

def _create_indexes(sample_collection_name, frame_collection_name): (source) ¶

Undocumented

def _create_sample_document_cls(dataset, sample_collection_name, field_docs=None): (source) ¶

Undocumented

def _declare_fields(dataset, doc_cls, field_docs=None): (source) ¶

Undocumented

def _delete_dataset_doc(dataset_doc): (source) ¶

Undocumented

def _delete_non_persistent_datasets(verbose=False): (source) ¶

Undocumented

def _discard_none(values): (source) ¶

Undocumented

def _do_load_dataset(obj, name): (source) ¶

Undocumented

def _ensure_index(dataset, db_field, unique=False): (source) ¶

Undocumented

def _extract_archive_if_necessary(archive_path, cleanup): (source) ¶

Undocumented

def _finalize_frames(sample_collection, key_field, frame_key_field): (source) ¶

Undocumented

def _get_collstats(coll): (source) ¶

Undocumented

def _get_frame_ids(arg): (source) ¶

Undocumented

def _get_frame_ids_for_sample(sample): (source) ¶

Undocumented

def _get_frames_pipeline(sample_collection): (source) ¶

Undocumented

def _get_group_field(schema): (source) ¶

Undocumented

def _get_group_id(sample_or_group): (source) ¶

Undocumented

def _get_group_ids(arg): (source) ¶

Undocumented

def _get_indexes_to_skip(view, frames=False): (source) ¶

Undocumented

def _get_media_type(sample): (source) ¶

Undocumented

def _get_random_characters(n): (source) ¶

Undocumented

def _get_sample_ids(arg): (source) ¶

Undocumented

def _get_samples_pipeline(sample_collection): (source) ¶

Undocumented

def _get_single_index_map(coll): (source) ¶

Undocumented

def _handle_nested_fields(schema): (source) ¶

Undocumented

def _index_frames(dataset, key_field, frame_key_field): (source) ¶

Undocumented

def _list_datasets(include_private=False, glob_patt=None, tags=None): (source) ¶

Undocumented

def _list_datasets_info(include_private=False, glob_patt=None, tags=None): (source) ¶

Undocumented

def _list_datasets_query(include_private=False, glob_patt=None, tags=None): (source) ¶

Undocumented

def _load_clips_source_dataset(frame_collection_name): (source) ¶

Undocumented

def _load_dataset(obj, name, virtual=False): (source) ¶

Undocumented

def _make_frame_collection_name(sample_collection_name): (source) ¶

Undocumented

def _make_merge_samples_generator(dataset, samples, key_fcn, id_map, skip_existing=False, insert_new=True, fields=None, omit_fields=None, merge_lists=True, overwrite=True, expand_schema=True): (source) ¶

Undocumented

def _make_sample_collection_name(dataset_id, patches=False, frames=False, clips=False): (source) ¶

Undocumented

def _merge_dataset_doc(dataset, collection_or_doc, fields=None, omit_fields=None, expand_schema=True, merge_info=True, overwrite_info=False): (source) ¶

Undocumented

def _merge_docs(sample_collection, merge_lists=True, fields=None, omit_fields=None, delete_fields=None, overwrite=False, frames=False): (source) ¶

Undocumented

def _merge_label_list_field(doc, elem_field, overwrite=False): (source) ¶

Undocumented

def _merge_list_field(doc, list_field): (source) ¶

Undocumented

def _merge_samples_pipeline(src_collection, dst_dataset, key_field, skip_existing=False, insert_new=True, fields=None, omit_fields=None, merge_lists=True, overwrite=True): (source) ¶

Undocumented

def _merge_samples_python(dataset, samples, key_field='filepath', key_fcn=None, skip_existing=False, insert_new=True, fields=None, omit_fields=None, merge_lists=True, overwrite=True, expand_schema=True, dynamic=False, progress=None, num_samples=None): (source) ¶

Undocumented

def _parse_field_mapping(field_mapping): (source) ¶

Undocumented

def _parse_fields(field_names): (source) ¶

Undocumented

def _save_view(view, fields=None): (source) ¶

Undocumented

def _set_field_read_only(field_doc, read_only): (source) ¶

Undocumented

def _to_list(arg): (source) ¶

Undocumented

def _update_no_overwrite(d, dnew): (source) ¶

Undocumented

def _validate_dataset_name(name, skip=None): (source) ¶

Validates that the given dataset name is available.

Parameters
name	a dataset name
skip:`None`	an optional `Dataset` to ignore
Returns
the slug
Raises
`ValueError`	if the name is not available

_SUMMARY_FIELD_KEY: str = (source) ¶

Undocumented

Value

'_summary_field'