fiftyone.core.map.mapper.Mapper

class documentation

class Mapper(abc.ABC): (source)

Known subclasses: fiftyone.core.map.mapper.LocalMapper

Constructor: Mapper(batch_cls, num_workers, batch_size)

View In Hierarchy

Base class for mapping samples in parallel

Class Method create Create a new mapper instance
Method __init__ Undocumented
Method map_samples Applies map function to each sample and returns an iterator of the results.
Property batch_size Number of samples per worker batch
Property num_workers Number of workers to use
Static Method _handle_map_error Common error handling when a map error occurs
Method _map_samples of the results.
Instance Variable _batch_cls Undocumented
Instance Variable _batch_size Undocumented
Instance Variable _num_workers Undocumented
@classmethod
@abc.abstractmethod
def create(cls, *, config: focc.FiftyOneConfig, batch_cls: type[fomb.SampleBatch], num_workers: int | None = None, **__): (source)
overridden in fiftyone.core.map.process.ProcessMapper, fiftyone.core.map.threading.ThreadMapper

Create a new mapper instance

def __init__(self, batch_cls: type[fomb.SampleBatch], num_workers: int, batch_size: int | None = None): (source)

Undocumented

def map_samples(self, sample_collection: SampleCollection[T], map_fcn: Callable[[T], R], *, progress: bool | Literal['workers'] | None = None, save: bool = False, skip_failures: bool = True) -> Iterator[tuple[bson.ObjectId, R]]: (source)

Applies map function to each sample and returns an iterator of the results.

Parameters
sample_collection:SampleCollection[T]The sample collection to map.
map_fcn:Callable[[T], R]The map function to apply to each sample.
progress:Union[bool, Literal["workers"]]Whether or not and how to render progress.
save:bool, optionalWhether to save mutated samples mutated in the map function. Defaults to False.
skip_failures:bool, optionalWhether to gracefully continue without raising an error if the map function raises an exception for a sample. Defaults to True.
Returns
Iterator[tuple[bson.ObjectId, R]]Undocumented
Yields
Iterator[Tuple[bson.ObjectId, R]]
The sample ID and the result of
the map function for the sample.
@property
batch_size: int | None = (source)

Number of samples per worker batch

@property
num_workers: int = (source)

Number of workers to use

@staticmethod
def _handle_map_error(sample_id: bson.ObjectId, err: Exception, skip_failures: bool): (source)

Common error handling when a map error occurs

@abc.abstractmethod
def _map_samples(self, sample_collection: SampleCollection[T], map_fcn: Callable[[T], R], *, progress: bool | Literal['workers'] | None, save: bool, skip_failures: bool) -> Iterator[tuple[bson.ObjectId, Exception | None, R | None]]: (source)
overridden in fiftyone.core.map.mapper.LocalMapper

Applies map function to each sample batch and returns an iterator
of the results.

Parameters
sample_collection:SampleCollection[T]The sample collection to map.
map_fcn:Callable[[T], R]The map function to apply to each sample.
progress:Union[bool, Literal["workers"]]Whether or not and how to render progress.
save:bool, optionalWhether to save mutated samples mutated in the map function. Defaults to False.
skip_failures:bool, optionalWhether to gracefully continue without raising an error if the map function raises an exception for a sample. Defaults to True.
Returns
Iterator[tuple[bson.ObjectId, Exception | None, R | None]]Undocumented
Yields
MapSampleBatchesReturnType
The sample ID, the exception raised
(if any), and the result of the map function for the sample (if no exception).
_batch_cls = (source)

Undocumented

_batch_size = (source)

Undocumented

_num_workers = (source)

Undocumented