Catalano.MachineLearning.Exploration

Class QLearning

java.lang.Object

Catalano.MachineLearning.Exploration.QLearning

public class QLearning
extends java.lang.Object

The class provides implementation of Q-Learning algorithm, known as off-policy Temporal Difference control.

Constructor Summary

Constructors

Constructor and Description
QLearning(int states, int actions, IExplorationPolicy explorationPolicy, boolean randomize) Initializes a new instance of the QLearning class.

Method Summary

Modifier and Type	Method and Description
int	GetAction(int state) Get next action from the specified state.
int	getActions() Amount of possible actions.
double	getDiscountFactor() Get Discount factor for the expected summary reward.
IExplorationPolicy	getExplorationPolicy() Exploration policy.
double	getLearningRate() Get Learning Rate
int	getStates() Amount of possible states.
void	setDiscountFactor(double discountFactor) Discount factor for the expected summary reward.
void	setExplorationPolicy(IExplorationPolicy explorationPolicy) Policy, which is used to select actions.
void	setLearningRate(double learningRate) Learning rate, [0, 1].
void	UpdateState(int previousState, int action, double reward, int nextState) Update Q-function's value for the previous state-action pair.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

QLearning

public QLearning(int states,
                 int actions,
                 IExplorationPolicy explorationPolicy,
                 boolean randomize)

Initializes a new instance of the QLearning class.

Parameters:

states - Amount of possible states.

actions - Amount of possible actions.

explorationPolicy - Exploration policy.

randomize - Randomize action estimates or not.

Method Detail

getStates

public int getStates()

Amount of possible states.

Returns:

States

getActions

public int getActions()

Amount of possible actions.

Returns:

Actions

getExplorationPolicy

public IExplorationPolicy getExplorationPolicy()

Exploration policy.

Returns:

Exploration Policy

setExplorationPolicy

public void setExplorationPolicy(IExplorationPolicy explorationPolicy)

Policy, which is used to select actions.

Parameters:

explorationPolicy - Exploration Policy

getLearningRate

public double getLearningRate()

Get Learning Rate

Returns:

Learning Rate

setLearningRate

public void setLearningRate(double learningRate)

Learning rate, [0, 1]. The value determines the amount of updates Q-function receives during learning. The greater the value, the more updates the function receives. The lower the value, the less updates it receives.

Parameters:

learningRate -

getDiscountFactor

public double getDiscountFactor()

Get Discount factor for the expected summary reward.

Returns:

Discount Factor

setDiscountFactor

public void setDiscountFactor(double discountFactor)

Discount factor for the expected summary reward. The value serves as multiplier for the expected reward. So if the value is set to 1, then the expected summary reward is not discounted. If the value is getting smaller, then smaller amount of the expected reward is used for actions' estimates update.

Parameters:

discountFactor -

GetAction

public int GetAction(int state)

Get next action from the specified state.

Parameters:

state - Current state to get an action for.

Returns:

Returns the action for the state.

UpdateState

public void UpdateState(int previousState,
                        int action,
                        double reward,
                        int nextState)

Update Q-function's value for the previous state-action pair.

Parameters:

previousState - Previous state.

action - Action, which leads from previous to the next state.

reward - Reward value, received by taking specified action from previous state.

nextState - Next state.