fiftyone.utils.data.importers.DatasetImporter

class documentation

class DatasetImporter(object): (source)

Known subclasses: fiftyone.utils.data.importers.BatchDatasetImporter, fiftyone.utils.data.importers.GenericSampleDatasetImporter, fiftyone.utils.data.importers.LabeledImageDatasetImporter, fiftyone.utils.data.importers.LabeledVideoDatasetImporter, fiftyone.utils.data.importers.UnlabeledImageDatasetImporter, fiftyone.utils.data.importers.UnlabeledMediaDatasetImporter, fiftyone.utils.data.importers.UnlabeledVideoDatasetImporter

Constructor: DatasetImporter(dataset_dir, shuffle, seed, max_samples)

View In Hierarchy

Base interface for importing datasets stored on disk into FiftyOne.

Typically, dataset importers should implement the parameters documented on this class, although this is not mandatory.

See :ref:`this page <writing-a-custom-dataset-importer>` for information about implementing/using dataset importers.

Parameters
dataset_dir	the dataset directory. This may be optional for some importers
shuffle	whether to randomly shuffle the order in which the samples are imported
seed	a random seed to use when shuffling
max_samples	a maximum number of samples to import. By default, all samples are imported

Method	`__enter__`	Undocumented
Method	`__exit__`	Undocumented
Method	`__init__`	Undocumented
Method	`__iter__`	Undocumented
Method	`__len__`	The total number of samples that will be imported.
Method	`__next__`	Returns information about the next sample in the dataset.
Method	`close`	Performs any necessary actions after the last sample has been imported.
Method	`get_dataset_info`	Returns the dataset info for the dataset.
Method	`setup`	Performs any necessary setup before importing the first sample in the dataset.
Instance Variable	`dataset_dir`	Undocumented
Instance Variable	`max_samples`	Undocumented
Instance Variable	`seed`	Undocumented
Instance Variable	`shuffle`	Undocumented
Property	`has_dataset_info`	Whether this importer produces a dataset info dictionary.
Method	`_preprocess_list`	Internal utility that preprocesses the given list---which is presumed to be a list defining the samples that should be imported---by applying the values of the `shuffle`, `seed`, and `max_samples` parameters of the importer.

def __enter__(self): (source) ¶

Undocumented

def __exit__(self, *args): (source) ¶

Undocumented

def __init__(self, dataset_dir=None, shuffle=False, seed=None, max_samples=None): (source) ¶

overridden in fiftyone.utils.bdd.BDDDatasetImporter, fiftyone.utils.coco.COCODetectionDatasetImporter, fiftyone.utils.csv.CSVDatasetImporter, fiftyone.utils.cvat.CVATImageDatasetImporter, fiftyone.utils.cvat.CVATVideoDatasetImporter, fiftyone.utils.data.importers.FiftyOneDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageClassificationDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageDetectionDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageLabelsDatasetImporter, fiftyone.utils.data.importers.FiftyOneTemporalDetectionDatasetImporter, fiftyone.utils.data.importers.FiftyOneVideoLabelsDatasetImporter, fiftyone.utils.data.importers.ImageClassificationDirectoryTreeImporter, fiftyone.utils.data.importers.ImageDirectoryImporter, fiftyone.utils.data.importers.ImageSegmentationDirectoryImporter, fiftyone.utils.data.importers.LegacyFiftyOneDatasetImporter, fiftyone.utils.data.importers.MediaDirectoryImporter, fiftyone.utils.data.importers.VideoClassificationDirectoryTreeImporter, fiftyone.utils.data.importers.VideoDirectoryImporter, fiftyone.utils.data.ingestors.LabeledImageDatasetIngestor, fiftyone.utils.data.ingestors.LabeledVideoDatasetIngestor, fiftyone.utils.data.ingestors.UnlabeledImageDatasetIngestor, fiftyone.utils.data.ingestors.UnlabeledVideoDatasetIngestor, fiftyone.utils.dicom.DICOMDatasetImporter, fiftyone.utils.geojson.GeoJSONDatasetImporter, fiftyone.utils.geotiff.GeoTIFFDatasetImporter, fiftyone.utils.kitti.KITTIDetectionDatasetImporter, fiftyone.utils.openimages.OpenImagesDatasetImporter, fiftyone.utils.openlabel.OpenLABELImageDatasetImporter, fiftyone.utils.openlabel.OpenLABELVideoDatasetImporter, fiftyone.utils.places.PlacesDatasetImporter, fiftyone.utils.tf.TFRecordsLabeledImageDatasetImporter, fiftyone.utils.voc.VOCDetectionDatasetImporter, fiftyone.utils.yolo.YOLOv4DatasetImporter, fiftyone.utils.yolo.YOLOv5DatasetImporter

Undocumented

def __iter__(self): (source) ¶

overridden in fiftyone.utils.bdd.BDDDatasetImporter, fiftyone.utils.coco.COCODetectionDatasetImporter, fiftyone.utils.csv.CSVDatasetImporter, fiftyone.utils.cvat.CVATImageDatasetImporter, fiftyone.utils.cvat.CVATVideoDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageClassificationDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageDetectionDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageLabelsDatasetImporter, fiftyone.utils.data.importers.FiftyOneTemporalDetectionDatasetImporter, fiftyone.utils.data.importers.FiftyOneVideoLabelsDatasetImporter, fiftyone.utils.data.importers.ImageClassificationDirectoryTreeImporter, fiftyone.utils.data.importers.ImageDirectoryImporter, fiftyone.utils.data.importers.ImageSegmentationDirectoryImporter, fiftyone.utils.data.importers.LegacyFiftyOneDatasetImporter, fiftyone.utils.data.importers.MediaDirectoryImporter, fiftyone.utils.data.importers.VideoClassificationDirectoryTreeImporter, fiftyone.utils.data.importers.VideoDirectoryImporter, fiftyone.utils.data.ingestors.LabeledImageDatasetIngestor, fiftyone.utils.data.ingestors.LabeledVideoDatasetIngestor, fiftyone.utils.data.ingestors.UnlabeledImageDatasetIngestor, fiftyone.utils.data.ingestors.UnlabeledVideoDatasetIngestor, fiftyone.utils.dicom.DICOMDatasetImporter, fiftyone.utils.geojson.GeoJSONDatasetImporter, fiftyone.utils.geotiff.GeoTIFFDatasetImporter, fiftyone.utils.kitti.KITTIDetectionDatasetImporter, fiftyone.utils.openimages.OpenImagesDatasetImporter, fiftyone.utils.openlabel.OpenLABELImageDatasetImporter, fiftyone.utils.openlabel.OpenLABELVideoDatasetImporter, fiftyone.utils.places.PlacesDatasetImporter, fiftyone.utils.tf.TFRecordsLabeledImageDatasetImporter, fiftyone.utils.voc.VOCDetectionDatasetImporter, fiftyone.utils.yolo.YOLOv4DatasetImporter, fiftyone.utils.yolo.YOLOv5DatasetImporter

Undocumented

def __len__(self): (source) ¶

overridden in fiftyone.utils.bdd.BDDDatasetImporter, fiftyone.utils.coco.COCODetectionDatasetImporter, fiftyone.utils.csv.CSVDatasetImporter, fiftyone.utils.cvat.CVATImageDatasetImporter, fiftyone.utils.cvat.CVATVideoDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageClassificationDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageDetectionDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageLabelsDatasetImporter, fiftyone.utils.data.importers.FiftyOneTemporalDetectionDatasetImporter, fiftyone.utils.data.importers.FiftyOneVideoLabelsDatasetImporter, fiftyone.utils.data.importers.GroupDatasetImporter, fiftyone.utils.data.importers.ImageClassificationDirectoryTreeImporter, fiftyone.utils.data.importers.ImageDirectoryImporter, fiftyone.utils.data.importers.ImageSegmentationDirectoryImporter, fiftyone.utils.data.importers.LegacyFiftyOneDatasetImporter, fiftyone.utils.data.importers.MediaDirectoryImporter, fiftyone.utils.data.importers.VideoClassificationDirectoryTreeImporter, fiftyone.utils.data.importers.VideoDirectoryImporter, fiftyone.utils.data.ingestors.LabeledImageDatasetIngestor, fiftyone.utils.data.ingestors.LabeledVideoDatasetIngestor, fiftyone.utils.data.ingestors.UnlabeledImageDatasetIngestor, fiftyone.utils.data.ingestors.UnlabeledVideoDatasetIngestor, fiftyone.utils.dicom.DICOMDatasetImporter, fiftyone.utils.geojson.GeoJSONDatasetImporter, fiftyone.utils.geotiff.GeoTIFFDatasetImporter, fiftyone.utils.kitti.KITTIDetectionDatasetImporter, fiftyone.utils.openimages.OpenImagesDatasetImporter, fiftyone.utils.openlabel.OpenLABELImageDatasetImporter, fiftyone.utils.openlabel.OpenLABELVideoDatasetImporter, fiftyone.utils.places.PlacesDatasetImporter, fiftyone.utils.voc.VOCDetectionDatasetImporter, fiftyone.utils.yolo.YOLOv4DatasetImporter, fiftyone.utils.yolo.YOLOv5DatasetImporter

The total number of samples that will be imported.

Raises
`TypeError`	if the total number is not known

def __next__(self): (source) ¶

overridden in fiftyone.utils.data.importers.BatchDatasetImporter, fiftyone.utils.data.importers.GenericSampleDatasetImporter, fiftyone.utils.data.importers.LabeledImageDatasetImporter, fiftyone.utils.data.importers.LabeledVideoDatasetImporter, fiftyone.utils.data.importers.UnlabeledImageDatasetImporter, fiftyone.utils.data.importers.UnlabeledMediaDatasetImporter, fiftyone.utils.data.importers.UnlabeledVideoDatasetImporter

Returns information about the next sample in the dataset.

Returns
subclass-specific information for the sample
Raises
`StopIteration`	if there are no more samples to import

def close(self, *args): (source) ¶

overridden in fiftyone.utils.dicom.DICOMDatasetImporter, fiftyone.utils.tf.TFRecordsLabeledImageDatasetImporter

Performs any necessary actions after the last sample has been imported.

This method is called when the importer's context manager interface is exited, DatasetImporter.__exit__.

Parameters
*args	the arguments to `DatasetImporter.__exit__`

def get_dataset_info(self): (source) ¶

overridden in fiftyone.utils.coco.COCODetectionDatasetImporter, fiftyone.utils.cvat.CVATImageDatasetImporter, fiftyone.utils.cvat.CVATVideoDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageClassificationDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageDetectionDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageLabelsDatasetImporter, fiftyone.utils.data.importers.FiftyOneTemporalDetectionDatasetImporter, fiftyone.utils.data.importers.FiftyOneVideoLabelsDatasetImporter, fiftyone.utils.data.importers.ImageClassificationDirectoryTreeImporter, fiftyone.utils.data.importers.LegacyFiftyOneDatasetImporter, fiftyone.utils.data.importers.VideoClassificationDirectoryTreeImporter, fiftyone.utils.openimages.OpenImagesDatasetImporter, fiftyone.utils.openlabel.OpenLABELImageDatasetImporter, fiftyone.utils.openlabel.OpenLABELVideoDatasetImporter, fiftyone.utils.yolo.YOLOv4DatasetImporter, fiftyone.utils.yolo.YOLOv5DatasetImporter

Returns the dataset info for the dataset.

By convention, this method should be called after all samples in the dataset have been imported.

Returns
a dict of dataset info

def setup(self): (source) ¶

Performs any necessary setup before importing the first sample in the dataset.

This method is called when the importer's context manager interface is entered, DatasetImporter.__enter__.

dataset_dir: None = (source) ¶

Undocumented

max_samples: None = (source) ¶

Undocumented

seed: None = (source) ¶

Undocumented

shuffle: False = (source) ¶

Undocumented

@property
has_dataset_info = (source) ¶

overridden in fiftyone.utils.bdd.BDDDatasetImporter, fiftyone.utils.coco.COCODetectionDatasetImporter, fiftyone.utils.csv.CSVDatasetImporter, fiftyone.utils.cvat.CVATImageDatasetImporter, fiftyone.utils.cvat.CVATVideoDatasetImporter, fiftyone.utils.data.importers.BatchDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageClassificationDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageDetectionDatasetImporter, fiftyone.utils.data.importers.FiftyOneImageLabelsDatasetImporter, fiftyone.utils.data.importers.FiftyOneTemporalDetectionDatasetImporter, fiftyone.utils.data.importers.FiftyOneVideoLabelsDatasetImporter, fiftyone.utils.data.importers.ImageClassificationDirectoryTreeImporter, fiftyone.utils.data.importers.ImageDirectoryImporter, fiftyone.utils.data.importers.ImageSegmentationDirectoryImporter, fiftyone.utils.data.importers.LegacyFiftyOneDatasetImporter, fiftyone.utils.data.importers.MediaDirectoryImporter, fiftyone.utils.data.importers.VideoClassificationDirectoryTreeImporter, fiftyone.utils.data.importers.VideoDirectoryImporter, fiftyone.utils.data.ingestors.LabeledImageDatasetIngestor, fiftyone.utils.data.ingestors.LabeledVideoDatasetIngestor, fiftyone.utils.data.ingestors.UnlabeledImageDatasetIngestor, fiftyone.utils.data.ingestors.UnlabeledVideoDatasetIngestor, fiftyone.utils.dicom.DICOMDatasetImporter, fiftyone.utils.geojson.GeoJSONDatasetImporter, fiftyone.utils.geotiff.GeoTIFFDatasetImporter, fiftyone.utils.kitti.KITTIDetectionDatasetImporter, fiftyone.utils.openimages.OpenImagesDatasetImporter, fiftyone.utils.openlabel.OpenLABELImageDatasetImporter, fiftyone.utils.openlabel.OpenLABELVideoDatasetImporter, fiftyone.utils.places.PlacesDatasetImporter, fiftyone.utils.tf.TFRecordsLabeledImageDatasetImporter, fiftyone.utils.voc.VOCDetectionDatasetImporter, fiftyone.utils.yolo.YOLOv4DatasetImporter, fiftyone.utils.yolo.YOLOv5DatasetImporter

Whether this importer produces a dataset info dictionary.

def _preprocess_list(self, l): (source) ¶

Internal utility that preprocesses the given list---which is presumed to be a list defining the samples that should be imported---by applying the values of the shuffle, seed, and max_samples parameters of the importer.

You may also provide an iterable, in which case the output will also be an iterable, unless the elements must be shuffled, in which case the iterable must be read in-memory into a list and returned as a list.

Parameters
l	a list or iterable
Returns
a processed copy of the list/iterable