fiftyone.utils.eval.classification.BinaryEvaluation

class documentation

class BinaryEvaluation(ClassificationEvaluation): (source)

Constructor: BinaryEvaluation(config)

Binary classification evaluation.

Evaluates the binary classification predictions in the given collection with respect to the specified ground truth labels.

Any missing ground truth or prediction labels are assumed to be examples of the negative class (with zero confidence, for predictions).

If an eval_key is specified, this method will record the TP/FP/FN/TN status of each prediction in this field.

This method requires that the (neg_label, pos_label) label strings for the task are provided via the classes argument.

Parameters
config	a `BinaryEvaluationConfig`

Method	`evaluate_samples`	Evaluates the predicted classifications in the given samples with respect to the specified ground truth labels.
Method	`register_samples`	Registers the collection on which evaluation will be performed.

Inherited from ClassificationEvaluation:

Method	`cleanup`	Cleans up the results of the run with the given key from the collection.
Method	`get_fields`	Gets the fields that were involved in the given run.
Method	`rename`	Performs any necessary operations required to rename this run's key.
Method	`_validate_run`	Subclass-specific validation when a run with the given key already exists.

Inherited from BaseEvaluationMethod (via ClassificationEvaluation):

Method	`cleanup_custom_metrics`	Undocumented
Method	`compute_custom_metrics`	Undocumented
Method	`get_custom_metric_fields`	Undocumented
Method	`rename_custom_metrics`	Undocumented
Method	`_get_custom_metrics`	Undocumented

Inherited from EvaluationMethod (via ClassificationEvaluation, BaseEvaluationMethod):

Class Method	`run_info_cls`	The `BaseRunInfo` class associated with this class.
Class Method	`_patch_function`	A function that can patch any ReferenceField issues with a dataset's runs.
Class Method	`_results_cache_field`	The `fiftyone.core.dataset.Dataset` field that stores the results cache for these runs.
Class Method	`_run_str`	A string to use when referring to these runs in log messages.
Class Method	`_runs_field`	The `fiftyone.core.odm.dataset.DatasetDocument` field in which these runs are stored.

Inherited from BaseRun (via ClassificationEvaluation, BaseEvaluationMethod, EvaluationMethod):

Class Method	`delete_run`	Deletes the results associated with the given run key from the collection.
Class Method	`delete_runs`	Deletes all runs from the collection.
Class Method	`get_run_info`	Gets the `BaseRunInfo` for the given key on the collection.
Class Method	`has_cached_run_results`	Determines whether `BaseRunResults` for the given key are cached on the collection.
Class Method	`list_runs`	Returns the list of run keys on the given collection.
Class Method	`load_run_results`	Loads the `BaseRunResults` for the given key on the collection.
Class Method	`load_run_view`	Loads the `fiftyone.core.view.DatasetView` on which the specified run was performed.
Class Method	`save_run_info`	Saves the run information on the collection.
Class Method	`save_run_results`	Saves the run results on the collection.
Class Method	`update_run_config`	Updates the `BaseRunConfig` for the given run on the collection.
Class Method	`update_run_key`	Replaces the key for the given run with a new key.
Method	`ensure_requirements`	Ensures that any necessary packages to execute this run are installed.
Method	`ensure_usage_requirements`	Ensures that any necessary packages to use existing results for this run are installed.
Method	`register_run`	Registers a run of this method under the given key on the given collection.
Method	`validate_run`	Validates that the collection can accept this run.
Class Method	`_get_run_doc`	Undocumented
Class Method	`_get_run_docs`	Undocumented
Class Method	`_get_run_fields`	Undocumented
Method	`_validate_fields_match`	Undocumented

Inherited from Configurable (via ClassificationEvaluation, BaseEvaluationMethod, EvaluationMethod, BaseRun):

Method	`__init__`	Undocumented
Instance Variable	`config`	Undocumented

def evaluate_samples(self, samples, eval_key=None, classes=None, missing=None, progress=None): (source) ¶

overrides fiftyone.utils.eval.classification.ClassificationEvaluation.evaluate_samples

Evaluates the predicted classifications in the given samples with respect to the specified ground truth labels.

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
eval_key:`None`	an evaluation key for this evaluation
classes:`None`	the list of possible classes. If not provided, the observed ground truth/predicted labels are used for results purposes
missing:`None`	a missing label string. Any None-valued labels are given this label for results purposes
progress:`None`	whether to render a progress bar (True/False), use the default value `fiftyone.config.show_progress_bars` (None), or a progress callback function to invoke instead
Returns
a `ClassificationResults` instance

def register_samples(self, samples, eval_key): (source) ¶

overrides fiftyone.utils.eval.classification.ClassificationEvaluation.register_samples

Registers the collection on which evaluation will be performed.

This method will be called before calling evaluate_samples. Subclasses can extend this method to perform any setup required for an evaluation run.

Parameters
samples	a `fiftyone.core.collections.SampleCollection`
eval_key	the evaluation key for this evaluation