MMLF package interface

Maja Machine Learning Framework

The Maja Machine Learning Framework (MMLF) is a general framework for problems in the domain of Reinforcement Learning (RL). It provides a set of RL related algorithms and a set of benchmark domains. Furthermore it is easily extensible and allows to automate benchmarking of different agents.

Among the RL algorithms are TD(lambda), DYNA-TD, CMA-ES, Fitted R-Max, and Monte-Carlo learning. MMLF contains different variants of the maze-world and pole-balancing problem class as well as the mountain-car testbed.

Further documentation is available under http://mmlf.sourceforge.net/

Contact the mailing list MMLF-support@lists.sourceforge.net if you have any questions.

exception mmlf.RWAreaInitializationFailedException

Exception raised if initialization of MMLF RW area failed.

mmlf.getRWPath()

Return the path of the MMLF RW area to be used.

This defaults to the path given by the environment variable $MMLF_RW_PATH if this variable exists. Otherwise, $HOME/.mmlf is used if the environment variable $HOME exists (typically under unix-like OS). Under Windows OS, $USERPROFILE/.mmlf is used instead. If None of these variable exists, an Exception of type RWAreaInitializationFailedException is raised.

mmlf.initializeRWArea(rwPath=None)

Initialize the RW area.

If rwPath is specified, the RW area located under that path is used. Otherwise, the standard RW area returned by getRWPath() is used.

If the required RW area does not exist, it is automatically created under the specified path. If config files in the RW area are missing, they are automatically restored.

mmlf.loadExperimentConfig(experimentPath)

Load the specification of an MMLF experiment.

Loads the specification of an MMLF experiment from the file experiment_config.yaml in experimentPath. Returns a dictionary containing the experiment’s specification.

mmlf.loadWorld(worldConfigObject, baseUserDir=None, useGUI=False, keepObservers=[])

Load the world specified in the worldConfigObject.

worldConfigObject must be a dictionary defining the configuration of a world. This dictionary must have the following keys:

worldPackage:: The name of the subpackage of mmlf/world in which the world is located. environment:: Definition of the environment. Must be a dictionary with the keys “moduleName” (the python module in which the environment is defined; looked up in the subdirectory environments of the world package) and “configDict” (the config dictionary of the environment, must have the same key-values as the environment’s DEFAULT_CONFIG_DICT). agents:: Definition of the agent. Must be a dictionary with the keys “moduleName” (the python module in which the agent is defined; looked up in mmlf/agents or in the subdirectory agents of the world package) and “configDict” (the config dictionary of the agent, must have the same key-values as the agent’s DEFAULT_CONFIG_DICT). monitor:: The configuration dictionary of the MMLF monitor.

You may take a look at the Learn more about Experiments/Scripting experiments section of the MMLF documentation for an example.

baseUserDir must be a MMLF BaseUserDirectory object (which represent the MMLF RW area internally). If none, a new BaseUserDirectory object is created.

useGUI indicates whether we are running a graphical user interface. If True, this causes the agent and environment of the world to create Viewers (and thus requires that PyQt4 is installed).

Per default, loadWorld() removes all existing observer objects since these stem from prior runs. Of observables should be kept, these can be passed via the parameter keepObservers (a list).

mmlf.loadWorldFromConfigFile(configPath, useGUI=False)

Load the world specified in the configPath.

This method loads the MMLF world specified in the YAML-file under configPath. configPath can either be a relative path (in which case it is looked up in the config directory of MMLF RW area or an absolute path. The created MMLF World object is returned.

useGUI indicates whether we are running a graphical user interface. If True, this causes the agent and environment of the world to create Viewers (and thus requires that PyQt4 is installed).

mmlf.runExperiment(experimentConf)

Execute the experiment specified in experimentConf.

experimentConf must be a dictionary defining the specification of the experiment. This dictionary must have the following keys:

concurrency:: “Concurrent” if several runs of the experiment should be executed at the same time, “Sequential” if one run after the other should be executed. parallelProcesses:  : If concurrent execution is requested, this parameter specifies how many runs can be run in parallel. This value is upper-bounded by the number of cores. episodesPerRun:: Integer defining how many episodes each run of a world should last. runsPerWorld:: How many independent runs should be executed for each world. worlds:: A dictionary mapping world name to a world configuration dictionary. See loadWorld for more details about world configuration dictionaries.

You may take a look at the Learn more about Experiments/Scripting experiments section of the MMLF documentation for an example.

mmlf.setupConsoleLogging(*args, **kwargs)

Initializes the logging of messages to the console.

mmlf.setupFileLogging(logdir, *args, **kwargs)

Initializes the logging of messages into a log file in logdir.