Read Mode¶

When mlrl-testbed operates in so-called “read mode”, it reads the output data produced by a previously run experiment and allows to print it on the console or write it to different output files. Similar to the run mode, this mode of operation is based on the metadata.yml file that has been saved by a previous experiment.

Note

A metadata.yml file is saved by an experiment if any output data is saved or if the argument --save-meta-data true is specified explicitly (see here for more details).

The command below shows how the read mode can be enabled via the argument --mode read. This requires specifying an absolute or relative path to a directory that contains the aforementioned metadata.yml file via the argument --input-dir. In the given example, the evaluation results of the previously run experiment are printed (--print-evaluation true) and saved to output files (--save-evaluation true) in a given --base-dir. Other output data can be printed or saved via the respective arguments described here.

BOOMER

mlrl-testbed mlrl.boosting \
    --mode read \
    --input-dir path/to/experiment/
    --base-dir some/arbitrary/path/
    --print-evaluation true
    --save-evaluation true

SeCo

mlrl-testbed mlrl.seco \
    --mode read \
    --input-dir path/to/experiment/
    --base-dir some/arbitrary/path/
    --print-evaluation true
    --save-evaluation true

Of course, only data that has been saved to output files when the experiment was run originally can be accessed in read mode. If you are interested in data that has not been saved previously, you can use run mode to run experiments again. For this reason, a few arguments or some of their options listed here may not be available in read mode, as is noted under the mentioned link.

The read mode is particularly useful for inspecting the experimental results obtained for multiple experiments via batch mode. If the meta-data that is provided to the read mode consists of multiple experiments, the output data of all these experiments will be printed or saved. When using the arguments --print-evaluation or --save-evaluation, not only the evaluation results for individual experiments are printed or saved, but results for different folds are automatically aggregated and results for different experiments are condensed into a single output.