fiftyone.utils.eval.segmentation

module documentation

(source)

Segmentation evaluation.

Class	`SegmentationEvaluation`	Base class for segmentation evaluation methods.
Class	`SegmentationEvaluationConfig`	Base class for configuring `SegmentationEvaluation` instances.
Class	`SegmentationResults`	Class that stores the results of a segmentation evaluation.
Class	`SimpleEvaluation`	Standard pixelwise segmentation evaluation.
Class	`SimpleEvaluationConfig`	Class for configuring `SimpleEvaluation` instances.
Function	`evaluate_segmentations`	Evaluates the specified semantic segmentation masks in the given collection with respect to the specified ground truth masks.
Variable	`logger`	Undocumented
Function	`_compute_accuracy_precision_recall`	Undocumented
Function	`_compute_dice_score`	Undocumented
Function	`_compute_pixel_confusion_matrix`	Undocumented
Function	`_extract_contour_band_values`	Undocumented
Function	`_get_mask_values`	Undocumented
Function	`_parse_config`	Undocumented

def evaluate_segmentations(samples, pred_field, gt_field='ground_truth', eval_key=None, mask_targets=None, method=None, custom_metrics=None, progress=None, **kwargs): (source) ¶

Evaluates the specified semantic segmentation masks in the given collection with respect to the specified ground truth masks.

If the size of a predicted mask does not match the ground truth mask, it is resized to match the ground truth.

By default, this method simply performs pixelwise evaluation of the full masks, but other strategies such as boundary-only evaluation can be configured by passing additional parameters for the method's config class as kwargs.

The natively provided method values and their associated configs are:

"simple": SimpleEvaluationConfig

If an eval_key is provided, the accuracy, precision, and recall of each sample is recorded in top-level fields of each sample:

 Accuracy: sample.<eval_key>_accuracy
Precision: sample.<eval_key>_precision
   Recall: sample.<eval_key>_recall

In addition, when evaluating frame-level masks, the accuracy, precision, and recall of each frame if recorded in the following frame-level fields:

 Accuracy: frame.<eval_key>_accuracy
Precision: frame.<eval_key>_precision
   Recall: frame.<eval_key>_recall

Note

The mask values 0 and #000000 are treated as a background class for the purposes of computing evaluation metrics like precision and recall.

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
pred_field	the name of the field containing the predicted `fiftyone.core.labels.Segmentation` instances
gt_field:"ground_truth"	the name of the field containing the ground truth `fiftyone.core.labels.Segmentation` instances
eval_key:`None`	an evaluation key to use to refer to this evaluation
mask_targets:`None`	a dict mapping pixel values or RGB hex strings to labels. If not provided, the observed values are used as labels
method:`None`	a string specifying the evaluation method to use. The supported values are `fo.evaluation_config.segmentation_backends.keys()` and the default is `fo.evaluation_config.default_segmentation_backend`
custom_metrics:`None`	an optional list of custom metrics to compute or dict mapping metric names to kwargs dicts
progress:`None`	whether to render a progress bar (True/False), use the default value `fiftyone.config.show_progress_bars` (None), or a progress callback function to invoke instead
**kwargs	optional keyword arguments for the constructor of the `SegmentationEvaluationConfig` being used
Returns
a `SegmentationResults`

logger = (source) ¶

Undocumented

def _compute_accuracy_precision_recall(confusion_matrix, values, average): (source) ¶

Undocumented

def _compute_dice_score(confusion_matrix): (source) ¶

Undocumented

def _compute_pixel_confusion_matrix(pred_mask, gt_mask, values, bandwidth=None): (source) ¶

Undocumented

def _extract_contour_band_values(pred_mask, gt_mask, bandwidth): (source) ¶

Undocumented

def _get_mask_values(samples, pred_field, gt_field, progress=None): (source) ¶

Undocumented

def _parse_config(pred_field, gt_field, method, **kwargs): (source) ¶

Undocumented