fiftyone.core.map.mapper.LocalMapper

class documentation

class LocalMapper(Mapper, abc.ABC): (source)

Known subclasses: fiftyone.core.map.process.ProcessMapper, fiftyone.core.map.threading.ThreadMapper

Constructor: LocalMapper(batch_cls, num_workers, batch_size)

View In Hierarchy

Base class for mapping samples in parallelizing on the same machine

Method _map_samples of the results.
Method _map_samples_multiple_workers of the results.

Inherited from Mapper:

Class Method create Create a new mapper instance
Method __init__ Undocumented
Method map_samples Applies map function to each sample and returns an iterator of the results.
Property batch_size Number of samples per worker batch
Property num_workers Number of workers to use
Static Method _handle_map_error Common error handling when a map error occurs
Instance Variable _batch_cls Undocumented
Instance Variable _batch_size Undocumented
Instance Variable _num_workers Undocumented
def _map_samples(self, sample_collection: SampleCollection[T], map_fcn: Callable[[T], R], *, progress: bool | Literal['workers'] | None = None, save: bool = False, skip_failures: bool = True) -> Iterator[tuple[bson.ObjectId, R]]: (source)
overrides fiftyone.core.map.mapper.Mapper._map_samples

Applies map function to each sample batch and returns an iterator
of the results.

Parameters
sample_collection:SampleCollection[T]The sample collection to map.
map_fcn:Callable[[T], R]The map function to apply to each sample.
progress:Union[bool, Literal["workers"]]Whether or not and how to render progress.
save:bool, optionalWhether to save mutated samples mutated in the map function. Defaults to False.
skip_failures:bool, optionalWhether to gracefully continue without raising an error if the map function raises an exception for a sample. Defaults to True.
Returns
Iterator[tuple[bson.ObjectId, R]]Undocumented
Yields
MapSampleBatchesReturnType
The sample ID, the exception raised
(if any), and the result of the map function for the sample (if no exception).
@abc.abstractmethod
def _map_samples_multiple_workers(self, sample_collection: SampleCollection[T], map_fcn: Callable[[T], R], *, progress: bool | Literal['workers'] | None, save: bool, skip_failures: bool) -> Iterator[tuple[bson.ObjectId, Exception | None, R | None]]: (source)
overridden in fiftyone.core.map.process.ProcessMapper, fiftyone.core.map.threading.ThreadMapper

Applies map function to each sample batch and returns an iterator
of the results.

Parameters
sample_collection:SampleCollection[T]The sample collection to map.
map_fcn:Callable[[T], R]The map function to apply to each sample.
progress:Union[bool, Literal["workers"]]Whether or not and how to render progress.
save:bool, optionalWhether to save mutated samples mutated in the map function. Defaults to False.
skip_failures:bool, optionalWhether to gracefully continue without raising an error if the map function raises an exception for a sample. Defaults to True.
Returns
Iterator[tuple[bson.ObjectId, Exception | None, R | None]]Undocumented
Yields
MapSampleBatchesReturnType
The sample ID, the exception raised
(if any), and the result of the map function for the sample (if no exception).