fiftyone.utils.eval.detection

module documentation

(source)

Detection evaluation.

Class	`DetectionEvaluation`	Base class for detection evaluation methods.
Class	`DetectionEvaluationConfig`	Base class for configuring `DetectionEvaluation` instances.
Class	`DetectionResults`	Class that stores the results of a detection evaluation.
Function	`evaluate_detections`	Evaluates the predicted detections in the given samples with respect to the specified ground truth detections.
Variable	`logger`	Undocumented
Function	`_parse_config`	Undocumented
Function	`_tally_matches`	Undocumented

def evaluate_detections(samples, pred_field, gt_field='ground_truth', eval_key=None, classes=None, missing=None, method=None, iou=0.5, use_masks=False, use_boxes=False, classwise=True, dynamic=True, custom_metrics=None, progress=None, **kwargs): (source) ¶

Evaluates the predicted detections in the given samples with respect to the specified ground truth detections.

This method supports evaluating the following spatial data types:

Object detections in fiftyone.core.labels.Detections format
Instance segmentations in fiftyone.core.labels.Detections format with their mask attributes populated
Polygons in fiftyone.core.labels.Polylines format
Keypoints in fiftyone.core.labels.Keypoints format
Temporal detections in fiftyone.core.labels.TemporalDetections format

For spatial object detection evaluation, this method uses COCO-style evaluation by default.

When evaluating keypoints, "IoUs" are computed via object keypoint similarity.

For temporal segment detection, this method uses ActivityNet-style evaluation by default.

You can use the method parameter to select a different method, and you can optionally customize the method by passing additional parameters for the method's config class as kwargs.

The natively provided method values and their associated configs are:

"coco": fiftyone.utils.eval.coco.COCOEvaluationConfig
"open-images": fiftyone.utils.eval.openimages.OpenImagesEvaluationConfig
"activitynet": fiftyone.utils.eval.activitynet.ActivityNetEvaluationConfig

If an eval_key is provided, a number of fields are populated at the object- and sample-level recording the results of the evaluation:

True positive (TP), false positive (FP), and false negative (FN) counts for each sample are saved in top-level fields of each sample:
```
TP: sample.<eval_key>_tp
FP: sample.<eval_key>_fp
FN: sample.<eval_key>_fn
```
In addition, when evaluating frame-level objects, TP/FP/FN counts are recorded for each frame:
```
TP: frame.<eval_key>_tp
FP: frame.<eval_key>_fp
FN: frame.<eval_key>_fn
```
The fields listed below are populated on each individual object; these fields tabulate the TP/FP/FN status of the object, the ID of the matching object (if any), and the matching IoU:
```
TP/FP/FN: object.<eval_key>
      ID: object.<eval_key>_id
     IoU: object.<eval_key>_iou
```

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
pred_field	the name of the field containing the predicted `fiftyone.core.labels.Detections`, `fiftyone.core.labels.Polylines`, `fiftyone.core.labels.Keypoints`, or `fiftyone.core.labels.TemporalDetections`
gt_field:"ground_truth"	the name of the field containing the ground truth `fiftyone.core.labels.Detections`, `fiftyone.core.labels.Polylines`, `fiftyone.core.labels.Keypoints`, or `fiftyone.core.labels.TemporalDetections`
eval_key:`None`	an evaluation key to use to refer to this evaluation
classes:`None`	the list of possible classes. If not provided, the observed ground truth/predicted labels are used
missing:`None`	a missing label string. Any unmatched objects are given this label for results purposes
method:`None`	a string specifying the evaluation method to use. The supported values are `fo.evaluation_config.detection_backends.keys()` and the default is `fo.evaluation_config.default_detection_backend`
iou:0.50	the IoU threshold to use to determine matches
use_masks:`False`	whether to compute IoUs using the instances masks in the `mask` attribute of the provided objects, which must be `fiftyone.core.labels.Detection` instances
use_boxes:`False`	whether to compute IoUs using the bounding boxes of the provided `fiftyone.core.labels.Polyline` instances rather than using their actual geometries
classwise:`True`	whether to only match objects with the same class label (True) or allow matches between classes (False)
dynamic:`True`	whether to declare the dynamic object-level attributes that are populated on the dataset's schema
custom_metrics:`None`	an optional list of custom metrics to compute or dict mapping metric names to kwargs dicts
progress:`None`	whether to render a progress bar (True/False), use the default value `fiftyone.config.show_progress_bars` (None), or a progress callback function to invoke instead
**kwargs	optional keyword arguments for the constructor of the `DetectionEvaluationConfig` being used
Returns
a `DetectionResults`

logger = (source) ¶

Undocumented

def _parse_config(pred_field, gt_field, method, is_temporal, **kwargs): (source) ¶

Undocumented

def _tally_matches(matches): (source) ¶

Undocumented