Observables

Module for that implements various kinds of observables

Currently implemented observables are

• FloatStreamObservable
• TrajectoryObservable
• StateActionValuesObservable
• FunctionOverStateSpaceObservable
• ModelObservable

class framework.observables.AllObservables

The set of observables is itself observable, too

addObservable(observable)

Add a new observable to the set of all observables.

getAllObservablesOfType(type)

Return all observables of a given type.

getObservable(observableName, type)

Return all observable of a given type with given observableName.

removeObservable(observable)

Remove an observable from the set of all observables.

class framework.observables.FloatStreamObservable(title, time_dimension_name, value_name)

An observable class that handles a stream of floats

addValue(time, value)

Add a new value for the given point in time to the observable.

class framework.observables.FunctionOverStateSpaceObservable(title, discreteValues)

Observable class for observing functions over the state space.

This class implements an observable for observing (one-dimensional) functions defined over the state space. This function can either implement an mapping f: S -> R (real numbers, *discreteValues*=False) or a mapping f: S -> C (some discrete finite set of values, *discreteValues*=True). An example for the first kind of function would be the optimal value function V(s), an example for the second case a deterministic policy pi(s), with C being the action space.

plot(function, fig, stateSpace, actionSpace, plotStateDims=None, rasterPoints=100)

Creates a graphical representation of a FunctionOverStateSpace.

Creates a plot of policy in the 2D subspace of the state space spanned by stateIndex1 and stateIndex2.

updateFunction(function)

Inform observable that the encapsulated function has changed.

class framework.observables.ModelObservable(title)

Observable class for observing models.

This class implements an observable for observing MMLF models. Whenever a new state transition, a new start state or a new terminal state is added to the model, this observable notifies all observers of this change.

plot(model, fig, stateSpace, colouring, plotSamples, minExplorationValue, plotStateDims, dimValues)

Does the actual plotting (either for viewer or for log-graphic).

updateModel(model)

Inform observable that the encapsulated model has changed.

class framework.observables.Observable(title)

Base class for all MMLf observables.

addObserver(observer)

Add an observer that gets informed of all changes of this observable.

removeObserver(observer)

Remove an observer of this observable.

class framework.observables.StateActionValuesObservable(title)

Observable class for real valued functions over the state action space.

This might be for instance a Q-function or the eligibility traces or a stochastic policy.

plot(function, actions, fig, stateSpace, plotStateDims=None, plotActions=None, rasterPoints=100)

Plots the q-Function for the case of a 2-dim subspace of the state space.

plotStateDims :The 2 dims that should be plotted plotActions : The action that should be plotted rasterPoints : How many raster points per dimension

updateValues(valueAccessFunction, actions)

Inform observable that the encapsulated function has changed.

class framework.observables.TrajectoryObservable(title)

An observable class that can monitor an agent’s trajectory

addTransition(state, action, reward, succState, episodeTerminated=False)

Add a new transition to the observable.

Adds the transition from state to succState for chosen action action and obtained reward reward to the observable. If episodeTerminated, succState is a terminal state of the environment.