Catalano.MachineLearning.Exploration

Class BoltzmannExploration

java.lang.Object

Catalano.MachineLearning.Exploration.BoltzmannExploration

All Implemented Interfaces:

IExplorationPolicy

public class BoltzmannExploration
extends java.lang.Object
implements IExplorationPolicy

The class implements exploration policy base on Boltzmann distribution. Acording to the policy, action a at state s is selected with the next probability: exp( Q( s, a ) / t ) p( s, a ) = ----------------------------- SUM( exp( Q( s, b ) / t ) ) b where Q(s, a) is action's a estimation (usefulness) at state s and t is Temperature.

Constructor Summary

Constructors

Constructor and Description
BoltzmannExploration(double temperature) Initializes a new instance of the BoltzmannExploration class.

Method Summary

Modifier and Type	Method and Description
int	ChooseAction(double[] actionEstimates) The method chooses an action depending on the provided estimates.
double	getTemperature() If temperature is low, then the policy tends to be more greedy.
void	setTemperature(double temperature) The property sets the balance between exploration and greedy actions.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BoltzmannExploration

public BoltzmannExploration(double temperature)

Initializes a new instance of the BoltzmannExploration class.

Parameters:

temperature - Temperature parameter of Boltzmann distribution.

Method Detail

getTemperature

public double getTemperature()

If temperature is low, then the policy tends to be more greedy.

Returns:

Temperature

setTemperature

public void setTemperature(double temperature)

The property sets the balance between exploration and greedy actions.

Parameters:

temperature - Temperature

ChooseAction

public int ChooseAction(double[] actionEstimates)

The method chooses an action depending on the provided estimates. The estimates can be any sort of estimate, which values usefulness of the action (expected summary reward, discounted reward, etc).

Specified by:

ChooseAction in interface IExplorationPolicy

Parameters:

actionEstimates - Action Estimates.

Returns:

Return selected action.