fiftyone.utils.eval.openimages.OpenImagesEvaluation

class documentation

class OpenImagesEvaluation(DetectionEvaluation): (source)

Constructor: OpenImagesEvaluation(config)

Open Images-style evaluation.

Parameters
config	a `OpenImagesEvaluationConfig`

Method	`__init__`	Undocumented
Method	`evaluate`	Performs Open Images-style evaluation on the given image.
Method	`generate_results`	Generates aggregate evaluation results for the samples.

Inherited from DetectionEvaluation:

Method	`cleanup`	Cleans up the results of the run with the given key from the collection.
Method	`get_fields`	Gets the fields that were involved in the given run.
Method	`register_samples`	Registers the collection on which evaluation will be performed.
Method	`rename`	Performs any necessary operations required to rename this run's key.
Instance Variable	`gt_field`	Undocumented
Instance Variable	`pred_field`	Undocumented
Method	`_validate_run`	Subclass-specific validation when a run with the given key already exists.

Inherited from BaseEvaluationMethod (via DetectionEvaluation):

Method	`cleanup_custom_metrics`	Undocumented
Method	`compute_custom_metrics`	Undocumented
Method	`get_custom_metric_fields`	Undocumented
Method	`rename_custom_metrics`	Undocumented
Method	`_get_custom_metrics`	Undocumented

Inherited from EvaluationMethod (via DetectionEvaluation, BaseEvaluationMethod):

Class Method	`run_info_cls`	The `BaseRunInfo` class associated with this class.
Class Method	`_patch_function`	A function that can patch any ReferenceField issues with a dataset's runs.
Class Method	`_results_cache_field`	The `fiftyone.core.dataset.Dataset` field that stores the results cache for these runs.
Class Method	`_run_str`	A string to use when referring to these runs in log messages.
Class Method	`_runs_field`	The `fiftyone.core.odm.dataset.DatasetDocument` field in which these runs are stored.

Inherited from BaseRun (via DetectionEvaluation, BaseEvaluationMethod, EvaluationMethod):

Class Method	`delete_run`	Deletes the results associated with the given run key from the collection.
Class Method	`delete_runs`	Deletes all runs from the collection.
Class Method	`get_run_info`	Gets the `BaseRunInfo` for the given key on the collection.
Class Method	`has_cached_run_results`	Determines whether `BaseRunResults` for the given key are cached on the collection.
Class Method	`list_runs`	Returns the list of run keys on the given collection.
Class Method	`load_run_results`	Loads the `BaseRunResults` for the given key on the collection.
Class Method	`load_run_view`	Loads the `fiftyone.core.view.DatasetView` on which the specified run was performed.
Class Method	`save_run_info`	Saves the run information on the collection.
Class Method	`save_run_results`	Saves the run results on the collection.
Class Method	`update_run_config`	Updates the `BaseRunConfig` for the given run on the collection.
Class Method	`update_run_key`	Replaces the key for the given run with a new key.
Method	`ensure_requirements`	Ensures that any necessary packages to execute this run are installed.
Method	`ensure_usage_requirements`	Ensures that any necessary packages to use existing results for this run are installed.
Method	`register_run`	Registers a run of this method under the given key on the given collection.
Method	`validate_run`	Validates that the collection can accept this run.
Class Method	`_get_run_doc`	Undocumented
Class Method	`_get_run_docs`	Undocumented
Class Method	`_get_run_fields`	Undocumented
Method	`_validate_fields_match`	Undocumented

Inherited from Configurable (via DetectionEvaluation, BaseEvaluationMethod, EvaluationMethod, BaseRun):

Instance Variable config Undocumented

def __init__(self, config): (source) ¶

overrides fiftyone.utils.eval.detection.DetectionEvaluation.__init__

Undocumented

def evaluate(self, sample_or_frame, eval_key=None): (source) ¶

overrides fiftyone.utils.eval.detection.DetectionEvaluation.evaluate

Performs Open Images-style evaluation on the given image.

Predicted objects are matched to ground truth objects in descending order of confidence, with matches requiring a minimum IoU of self.config.iou.

The self.config.classwise parameter controls whether to only match objects with the same class label (True) or allow matches between classes (False).

If a ground truth object has its self.config.iscrowd attribute set, then the object can have multiple true positive predictions matched to it.

Parameters
sample_or_frame	a `fiftyone.core.sample.Sample` or `fiftyone.core.frame.Frame`
eval_key:`None`	the evaluation key for this evaluation
Returns
a list of matched `(gt_label, pred_label, iou, pred_confidence, gt_id, pred_id)` tuples

def generate_results(self, samples, matches, eval_key=None, classes=None, missing=None, progress=None): (source) ¶

overrides fiftyone.utils.eval.detection.DetectionEvaluation.generate_results

Generates aggregate evaluation results for the samples.

This method generates precision and recall curves for the configured IoU at self.config.iou.

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
matches	a list of `(gt_label, pred_label, iou, pred_confidence, gt_id, pred_id)` matches. Either label can be `None` to indicate an unmatched object
eval_key:`None`	the evaluation key for this evaluation
classes:`None`	the list of possible classes. If not provided, the observed ground truth/predicted labels are used for results purposes
missing:`None`	a missing label string. Any unmatched objects are given this label for results purposes
progress:`None`	whether to render a progress bar (True/False), use the default value `fiftyone.config.show_progress_bars` (None), or a progress callback function to invoke instead
Returns
a `OpenImagesDetectionResults`