mlrl.testbed_sklearn.experiments.output.evaluation package

Author Michael Rapp (michael.rapp.ml@gmail.com)

Provides classes that allow to write evaluation results to different sinks.

class mlrl.testbed_sklearn.experiments.output.evaluation.ClassificationEvaluationDataExtractor

Bases: EvaluationDataExtractor

Obtains evaluation results according to classification evaluation measures.

class mlrl.testbed_sklearn.experiments.output.evaluation.EvaluationWriter(*extractors: EvaluationDataExtractor)

Bases: ResultWriter

Allows writing evaluation results to one or several sinks.

class InputExtractor(properties: TabularProperties, context: Context)

Bases: TabularDataExtractor

Uses TabularInputData that has previously been loaded via an input reader.

ALL_MEASURES = {<mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.data.OutputValue object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.evaluation.measures.Measure object>, <mlrl.testbed.experiments.output.data.OutputValue object>}
extract_data(state: ExperimentState, sinks: list[Sink]) → list[tuple[ExperimentState, OutputData]]

See mlrl.testbed.experiments.output.writer.DataExtractor.extract_data()

measurements: dict[DatasetType, dict[int, Measurements]] = {}
class mlrl.testbed_sklearn.experiments.output.evaluation.RankingEvaluationDataExtractor

Bases: EvaluationDataExtractor

Obtains evaluation results according to ranking evaluation measures.

class mlrl.testbed_sklearn.experiments.output.evaluation.RegressionEvaluationDataExtractor

Bases: EvaluationDataExtractor

Obtains evaluation results according to regression evaluation measures.

class mlrl.testbed_sklearn.experiments.output.evaluation.TabularEvaluationResult(measurements: Measurements)

Bases: EvaluationResult

Stores the evaluation results according to different measures.

KWARG_FOLD = 'fold_index'
OPTION_ACCURACY = 'accuracy'
OPTION_COVERAGE_ERROR = 'coverage_error'
OPTION_DISCOUNTED_CUMULATIVE_GAIN = 'dcg'
OPTION_EXAMPLE_WISE_F1 = 'example_wise_f1'
OPTION_EXAMPLE_WISE_JACCARD = 'example_wise_jaccard'
OPTION_EXAMPLE_WISE_PRECISION = 'example_wise_precision'
OPTION_EXAMPLE_WISE_RECALL = 'example_wise_recall'
OPTION_F1 = 'f1'
OPTION_HAMMING_ACCURACY = 'hamming_accuracy'
OPTION_HAMMING_LOSS = 'hamming_loss'
OPTION_JACCARD = 'jaccard'
OPTION_LABEL_RANKING_AVERAGE_PRECISION = 'lrap'
OPTION_MACRO_F1 = 'macro_f1'
OPTION_MACRO_JACCARD = 'macro_jaccard'
OPTION_MACRO_PRECISION = 'macro_precision'
OPTION_MACRO_RECALL = 'macro_recall'
OPTION_MEAN_ABSOLUTE_ERROR = 'mean_absolute_error'
OPTION_MEAN_ABSOLUTE_PERCENTAGE_ERROR = 'mean_absolute_percentage_error'
OPTION_MEAN_SQUARED_ERROR = 'mean_squared_error'
OPTION_MEDIAN_ABSOLUTE_ERROR = 'mean_absolute_error'
OPTION_MICRO_F1 = 'micro_f1'
OPTION_MICRO_JACCARD = 'micro_jaccard'
OPTION_MICRO_PRECISION = 'micro_precision'
OPTION_MICRO_RECALL = 'micro_recall'
OPTION_NORMALIZED_DISCOUNTED_CUMULATIVE_GAIN = 'ndcg'
OPTION_PRECISION = 'precision'
OPTION_PREDICTION_TIME = 'prediction_time'
OPTION_RANK_LOSS = 'rank_loss'
OPTION_RECALL = 'recall'
OPTION_SUBSET_ACCURACY = 'subset_accuracy'
OPTION_SUBSET_ZERO_ONE_LOSS = 'subset_zero_one_loss'
OPTION_TRAINING_TIME = 'training_time'
OPTION_ZERO_ONE_LOSS = 'zero_one_loss'
to_table(options: Options, **kwargs) → Table | None

See mlrl.testbed.experiments.output.data.TabularOutputData.to_table()

to_text(options: Options, **kwargs) → str | None

See mlrl.testbed.experiments.output.data.TextualOutputData.to_text()

Submodules