fiftyone.utils.eval.detection.DetectionEvaluation

class documentation

class DetectionEvaluation(BaseEvaluationMethod): (source)

Known subclasses: fiftyone.utils.eval.activitynet.ActivityNetEvaluation, fiftyone.utils.eval.coco.COCOEvaluation, fiftyone.utils.eval.openimages.OpenImagesEvaluation

Constructor: DetectionEvaluation(config)

View In Hierarchy

Base class for detection evaluation methods.

Parameters
config	a `DetectionEvaluationConfig`

Method	`__init__`	Undocumented
Method	`cleanup`	Cleans up the results of the run with the given key from the collection.
Method	`evaluate`	Evaluates the ground truth and predictions in the given document.
Method	`generate_results`	Generates aggregate evaluation results for the samples.
Method	`get_fields`	Gets the fields that were involved in the given run.
Method	`register_samples`	Registers the collection on which evaluation will be performed.
Method	`rename`	Performs any necessary operations required to rename this run's key.
Instance Variable	`gt_field`	Undocumented
Instance Variable	`pred_field`	Undocumented
Method	`_validate_run`	Subclass-specific validation when a run with the given key already exists.

Inherited from BaseEvaluationMethod:

Method	`cleanup_custom_metrics`	Undocumented
Method	`compute_custom_metrics`	Undocumented
Method	`get_custom_metric_fields`	Undocumented
Method	`rename_custom_metrics`	Undocumented
Method	`_get_custom_metrics`	Undocumented

Inherited from EvaluationMethod (via BaseEvaluationMethod):

Class Method	`run_info_cls`	The `BaseRunInfo` class associated with this class.
Class Method	`_patch_function`	A function that can patch any ReferenceField issues with a dataset's runs.
Class Method	`_results_cache_field`	The `fiftyone.core.dataset.Dataset` field that stores the results cache for these runs.
Class Method	`_run_str`	A string to use when referring to these runs in log messages.
Class Method	`_runs_field`	The `fiftyone.core.odm.dataset.DatasetDocument` field in which these runs are stored.

Inherited from BaseRun (via BaseEvaluationMethod, EvaluationMethod):

Class Method	`delete_run`	Deletes the results associated with the given run key from the collection.
Class Method	`delete_runs`	Deletes all runs from the collection.
Class Method	`get_run_info`	Gets the `BaseRunInfo` for the given key on the collection.
Class Method	`has_cached_run_results`	Determines whether `BaseRunResults` for the given key are cached on the collection.
Class Method	`list_runs`	Returns the list of run keys on the given collection.
Class Method	`load_run_results`	Loads the `BaseRunResults` for the given key on the collection.
Class Method	`load_run_view`	Loads the `fiftyone.core.view.DatasetView` on which the specified run was performed.
Class Method	`save_run_info`	Saves the run information on the collection.
Class Method	`save_run_results`	Saves the run results on the collection.
Class Method	`update_run_config`	Updates the `BaseRunConfig` for the given run on the collection.
Class Method	`update_run_key`	Replaces the key for the given run with a new key.
Method	`ensure_requirements`	Ensures that any necessary packages to execute this run are installed.
Method	`ensure_usage_requirements`	Ensures that any necessary packages to use existing results for this run are installed.
Method	`register_run`	Registers a run of this method under the given key on the given collection.
Method	`validate_run`	Validates that the collection can accept this run.
Class Method	`_get_run_doc`	Undocumented
Class Method	`_get_run_docs`	Undocumented
Class Method	`_get_run_fields`	Undocumented
Method	`_validate_fields_match`	Undocumented

Inherited from Configurable (via BaseEvaluationMethod, EvaluationMethod, BaseRun):

Instance Variable config Undocumented

def __init__(self, config): (source) ¶

overrides fiftyone.core.config.Configurable.__init__

overridden in fiftyone.utils.eval.activitynet.ActivityNetEvaluation, fiftyone.utils.eval.coco.COCOEvaluation, fiftyone.utils.eval.openimages.OpenImagesEvaluation

Undocumented

def cleanup(self, samples, eval_key): (source) ¶

overrides fiftyone.core.runs.BaseRun.cleanup

Cleans up the results of the run with the given key from the collection.

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
eval_key	Undocumented
key	a run key

def evaluate(self, doc, eval_key=None): (source) ¶

overridden in fiftyone.utils.eval.activitynet.ActivityNetEvaluation, fiftyone.utils.eval.coco.COCOEvaluation, fiftyone.utils.eval.openimages.OpenImagesEvaluation

Evaluates the ground truth and predictions in the given document.

Parameters
doc	a `fiftyone.core.document.Document`
eval_key:`None`	the evaluation key for this evaluation
Returns
a list of matched `(gt_label, pred_label, iou, pred_confidence)` tuples

def generate_results(self, samples, matches, eval_key=None, classes=None, missing=None, progress=None): (source) ¶

overridden in fiftyone.utils.eval.activitynet.ActivityNetEvaluation, fiftyone.utils.eval.coco.COCOEvaluation, fiftyone.utils.eval.openimages.OpenImagesEvaluation

Generates aggregate evaluation results for the samples.

Subclasses may perform additional computations here such as IoU sweeps in order to generate mAP, PR curves, etc.

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
matches	a list of `(gt_label, pred_label, iou, pred_confidence, gt_id, pred_id)` matches. Either label can be `None` to indicate an unmatched object
eval_key:`None`	the evaluation key for this evaluation
classes:`None`	the list of possible classes. If not provided, the observed ground truth/predicted labels are used for results purposes
missing:`None`	a missing label string. Any unmatched objects are given this label for results purposes
progress:`None`	whether to render a progress bar (True/False), use the default value `fiftyone.config.show_progress_bars` (None), or a progress callback function to invoke instead
Returns
a `DetectionResults`

def get_fields(self, samples, eval_key): (source) ¶

overrides fiftyone.core.runs.BaseRun.get_fields

Gets the fields that were involved in the given run.

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
eval_key	Undocumented
key	a run key
Returns
a list of fields

def register_samples(self, samples, eval_key, dynamic=True): (source) ¶

Registers the collection on which evaluation will be performed.

This method will be called before the first call to evaluate. Subclasses can extend this method to perform any setup required for an evaluation run.

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
eval_key	the evaluation key for this evaluation
dynamic:`True`	whether to declare the dynamic object-level attributes that are populated on the dataset's schema

def rename(self, samples, eval_key, new_eval_key): (source) ¶

overrides fiftyone.core.runs.BaseRun.rename

Performs any necessary operations required to rename this run's key.

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
eval_key	Undocumented
new_eval_key	Undocumented
key	a run key
new_key	a new run key

gt_field = (source) ¶

Undocumented

pred_field = (source) ¶

Undocumented

def _validate_run(self, samples, eval_key, existing_info): (source) ¶

overrides fiftyone.core.runs.BaseRun._validate_run

Subclass-specific validation when a run with the given key already exists.

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
eval_key	Undocumented
existing_info	a `BaseRunInfo`
key	a run key
Raises
`ValueError`	if the run is invalid