mlrl.testbed_sklearn.runnables module

Author: Michael Rapp (michael.rapp.ml@gmail.com)

Provides classes for running experiments using the scikit-learn framework.

class mlrl.testbed_sklearn.runnables.SkLearnEstimatorRunnable

Bases: SkLearnRunnable

An abstract base class for all programs that run an experiment using a specific scikit-learn estimator.

class EstimatorExtension(supported_classifiers: set[SklearnEstimator], supported_regressors: set[SklearnEstimator], supported_meta_classifiers: set[SklearnEstimator], supported_meta_regressors: set[SklearnEstimator], *dependencies: Extension)

Bases: Extension

An extension that configures the scikit-learn estimator to be used in an experiment.

static create_estimator_argument(supported_classifiers: set[SklearnEstimator], supported_regressors: set[SklearnEstimator], mode: ExperimentMode) → SetArgument

Creates and returns a SetArgument that allows to specify the name of the scikit-learn estimator to be used in an experiment.

Parameters:

  • supported_classifiers – A set that contains all supported scikit-learn classifiers

  • supported_regressors – A set that contains all supported scikit-learn regressors

  • mode – The mode of operation

Returns:

The SetArgument that has been created

static create_meta_estimator_argument(supported_meta_classifiers: set[SklearnEstimator], supported_meta_regressors: set[SklearnEstimator]) → SetArgument

Creates and returns a SetArgument that allows to specify the name of a scikit-learn meta-estimator to be used in an experiment.

Parameters:

  • supported_meta_classifiers – A set that contains all supported scikit-learn meta-classifiers

  • supported_meta_regressors – A set that contains all supported scikit-learn meta-regressors

Returns:

The SetArgument that has been created

get_supported_modes() → set[ExperimentMode]

See mlrl.testbed.extensions.extension.Extension.get_supported_modes()

create_classifier(mode: ExperimentMode, args: Namespace) → SkLearnClassifierMixin | None

See mlrl.testbed.runnables.Runnable.create_classifier()

create_regressor(mode: ExperimentMode, args: Namespace) → SkLearnRegressorMixin | None

See mlrl.testbed_sklearn.runnables.SkLearnRunnable.create_regressor()

get_algorithmic_arguments(mode: ExperimentMode, known_args: Namespace) → set[Argument]

See mlrl.testbed.runnables.Runnable.get_algorithmic_arguments()

get_extensions() → list[Extension]

See mlrl.testbed.runnables.Runnable.get_extensions()

class mlrl.testbed_sklearn.runnables.SkLearnRunnable

Bases: Runnable, ABC

An abstract base class for all programs that run an experiment using the scikit-learn framework.

class BatchConfigFile(file_path: str)

Bases: ConfigFile

A YAML configuration file that configures a batch of experiments using the scikit-learn framework to be run.

property dataset_args: list[ArgumentList]

See from mlrl.testbed.modes.BatchMode.ConfigFile.dataset_args()

class GlobalPredictorFactory(prediction_type: PredictionType)

Bases: PredictorFactory

Allow to create instances of type Predictor that obtain predictions from a global model.

create() → Predictor

See from mlrl.testbed_sklearn.experiments.problem_domain.SkLearnProblem.PredictorFactory.create()

class ProblemDomainExtension

Bases: Extension

An extension that configures the problem domain.

PROBLEM_TYPE = <mlrl.util.cli.SetArgument object>
static get_problem_domain(mode: ExperimentMode, args: Namespace, runnable: SkLearnRunnable, fit_kwargs: dict[str, Any] | None = None, predict_kwargs: dict[str, Any] | None = None) → ProblemDomain

Returns the problem domain that should be tackled by an experiment.

Parameters:

  • mode – The mode of operation

  • args – The command line arguments specified by the user

  • runnable – The SkLearnRunnable that is used to run the experiment

  • fit_kwargs – Optional keyword arguments to be passed to the estimator’s predict function

  • predict_kwargs – Optional keyword arguments to be passed to the estimator’s fit function

Returns:

The problem domain that should be tackled by the experiment

get_supported_modes() → set[ExperimentMode]

See mlrl.testbed.extensions.extension.Extension.get_supported_modes()

create_batch_config_file_factory() → Callable[[str], ConfigFile]

See mlrl.testbed.runnables.Runnable.create_batch_config_file_factory()

abstractmethod create_classifier(mode: ExperimentMode, args: Namespace) → SkLearnClassifierMixin | None

Must be implemented by subclasses in order to create a machine learning algorithm that can be applied to classification problems.

Parameters:

  • mode – The mode of operation

  • args – The command line arguments

Returns:

The learner that has been created or None, if regression problems are not supported

create_dataset_splitter(args: Namespace, load_dataset: bool = True) → DatasetSplitter

See mlrl.testbed.experiments.recipe.Recipe.create_dataset_splitter()

create_experiment_builder(experiment_mode: ExperimentMode, args: Namespace, command: Command, load_dataset: bool = True) → Builder

See mlrl.testbed.experiments.recipe.Recipe.create_experiment_builder()

create_predictor_factory(args: Namespace, prediction_type: PredictionType) → PredictorFactory

May be overridden by subclasses in order to create the SkLearnProblem.PredictorFactory that should be used for obtaining predictions from a previously trained model.

Parameters:

  • args – The command line arguments

  • prediction_type – The type of the predictions to be obtained

Returns:

The SkLearnProblem.PredictorFactory that has been created

create_problem_domain(mode: ExperimentMode, args: Namespace) → ProblemDomain

See mlrl.testbed.experiments.recipe.Recipe.create_problem_domain()

abstractmethod create_regressor(mode: ExperimentMode, args: Namespace) → SkLearnRegressorMixin | None

Must be implemented by subclasses in order to create a machine learning algorithm that can be applied to regression problems.

Parameters:

  • mode – The mode of operation

  • args – The command line arguments

Returns:

The learner that has been created or None, if regression problems are not supported

get_extensions() → list[Extension]

See mlrl.testbed.runnables.Runnable.get_extensions()

class mlrl.testbed_sklearn.runnables.SklearnEstimator(estimator_name: str, estimator_type: type[sklearn.base.BaseEstimator])

Bases: object

Represents a scikit-learn estimator that can be used with a SklearnEstimatorRunnable.

class SklearnArgument(name: str, parameter_name: str, arguments: list[Argument], description: str | None = None, type_hint: str | None = None)

Bases: Argument

A command line argument that allows to configure a hyperparameter of a scikit-learn estimator.

static from_type_name(argument_name: str, parameter_name: str, type_name: str, description: str | None = None) → SklearnArgument

Creates and returns an SklearnArgument from a given type name.

Parameters:

  • argument_name – The name to be used by the argument

  • parameter_name – the name of the hyperparameter

  • type_name – The type name to be parsed

  • description – An optional description of the argument

get_value(args: Namespace, default: Any | None = None) → Any | None

See mlrl.util.cli.Argument.get_value()

property algorithmic_arguments: set[SklearnArgument]

A set that contains the command line arguments that allow to control the hyperparameters of the estimator.

property can_be_default_instantiated: bool

True, if the estimator can be instantiated via a default constructor, False otherwise.

property can_be_instantiated_with_estimator: bool

True, if the estimator can be instantiated by providing another estimator as a constructor argument, False otherwise.

static get_supported_classifiers() → set[SklearnEstimator]

Returns a set that contains all supported scikit-learn classifiers.

Returns:

A set that contains the names of all supported classifiers

static get_supported_meta_classifiers() → set[SklearnEstimator]

Returns a set that contains all supported scikit-learn meta-classifiers.

Returns:

A set that contains the names of all supported meta-classifiers

static get_supported_meta_regressors() → set[SklearnEstimator]

Returns a set that contains all supported scikit-learn meta-regressors.

Returns:

A set that contains the names of all supported meta-regressors

static get_supported_regressors() → set[SklearnEstimator]

Returns a set that contains all supported scikit-learn regressors.

Returns:

A set that contains the names of all supported regressors

instantiate(args: Namespace | None = None, **kwargs) → sklearn.base.BaseEstimator

Creates and returns a new instance of the estimator.

Parameters:

  • args – Command line arguments specified by the user or None, if default hyperparameters should be used

  • kwargs – Optional keyword arguments to be passed to the estimator’s constructor

Returns:

The instance that has been created

property is_classifier: bool

True, if the estimator is a classifier, False otherwise.

property is_meta_estimator: bool

True, if the estimator is a meta-estimator, False otherwise.

property is_regressor: bool

True, if the estimator is a regressor, False otherwise.