mlrl.testbed.experiments.output.evaluation.measures module

Author Michael Rapp (michael.rapp.ml@gmail.com)

Provides classes for implementing evaluation measures.

class mlrl.testbed.experiments.output.evaluation.measures.Measure(option_key: str, name: str, evaluation_function: Callable[[Any, Any], float], percentage: bool = True, **kwargs)

Bases: OutputValue

An evaluation measure.

EvaluationFunction

alias of Callable[[Any, Any], float]

evaluate(ground_truth: Any, predictions: Any) → float

Applies the evaluation function to given predictions and the corresponding ground truth.

Parameters:

• ground_truth – A numpy.ndarray, scipy.sparse.spmatrix or scipy.sparse.sparray, shape (num_examples, num_outputs), that stores the ground truth

• predictions – A numpy.ndarray, scipy.sparse.spmatrix or scipy.sparse.sparray, shape (num_examples, num_outputs), that stores the predictions to be evaluated

Returns:

An evaluation score in [0, 1]