org.encog.neural.networks.training.propagation.quick

Class QuickPropagation

java.lang.Object

org.encog.ml.train.BasicTraining

org.encog.neural.networks.training.propagation.Propagation

org.encog.neural.networks.training.propagation.quick.QuickPropagation

All Implemented Interfaces:

MLTrain, BatchSize, LearningRate, GradientWorkerOwner, Train, MultiThreadable

public class QuickPropagation
extends Propagation
implements LearningRate

QPROP is an efficient training method that is based on Newton's Method. QPROP was introduced in a paper: An Empirical Study of Learning Speed in Back-Propagation Networks" (Scott E. Fahlman, 1988) http://www.heatonresearch.com/wiki/Quickprop

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	LAST_GRADIENTS Continuation tag for the last gradients.

Constructor Summary

Constructors

Constructor and Description
QuickPropagation(ContainsFlat network, MLDataSet training) Construct a QPROP trainer for flat networks.
QuickPropagation(ContainsFlat network, MLDataSet training, double theLearningRate) Construct a QPROP trainer for flat networks.

Method Summary

Modifier and Type	Method and Description
boolean	canContinue()
double[]	getLastDelta()
double	getLearningRate()
double	getOutputEpsilon()
double	getShrink()
void	initOthers() Perform training method specific init.
boolean	isValidResume(TrainingContinuation state) Determine if the specified continuation object is valid to resume with.
TrainingContinuation	pause() Pause the training.
void	resume(TrainingContinuation state) Resume training.
void	setBatchSize(int theBatchSize) Do not allow batch sizes other than 0, not supported.
void	setLearningRate(double rate) Set the learning rate, this is value is essentially a percent.
void	setOutputEpsilon(double theOutputEpsilon)
void	setShrink(double s)
double	updateWeight(double[] gradients, double[] lastGradient, int index) Update a weight.
double	updateWeight(double[] gradients, double[] lastGradient, int index, double dropoutRate) Update a weight with droput.

Methods inherited from class org.encog.neural.networks.training.propagation.Propagation

calculateGradients, finishTraining, finishTraining, fixFlatSpot, getBatchSize, getCurrentFlatNetwork, getDropoutRate, getL1, getL2, getLastGradient, getMethod, getThreadCount, iteration, iteration, report, rollIteration, setDroupoutRate, setErrorFunction, setL1, setL2, setThreadCount

Methods inherited from class org.encog.ml.train.BasicTraining

addStrategy, getError, getImplementationType, getIteration, getStrategies, getTraining, isTrainingDone, postIteration, preIteration, setError, setIteration, setTraining

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.encog.ml.train.MLTrain

addStrategy, getError, getImplementationType, getIteration, getStrategies, getTraining, isTrainingDone, setError, setIteration

Field Detail

LAST_GRADIENTS

public static final java.lang.String LAST_GRADIENTS

Continuation tag for the last gradients.

See Also:

Constant Field Values

Constructor Detail

QuickPropagation

public QuickPropagation(ContainsFlat network,
                        MLDataSet training)

Construct a QPROP trainer for flat networks. Uses a learning rate of 2.

Parameters:

network - The network to train.

training - The training data.

QuickPropagation

public QuickPropagation(ContainsFlat network,
                        MLDataSet training,
                        double theLearningRate)

Construct a QPROP trainer for flat networks.

Parameters:

network - The network to train.

training - The training data.

theLearningRate - The learning rate. 2 is a good suggestion as a learning rate to start with. If it fails to converge, then drop it. Just like backprop, except QPROP can take higher learning rates.

Method Detail

canContinue

public boolean canContinue()

Specified by:

canContinue in interface MLTrain

Returns:

True if the training can be paused, and later continued.

getLastDelta

public double[] getLastDelta()

Returns:

The last delta values.

getLearningRate

public double getLearningRate()

Specified by:

getLearningRate in interface LearningRate

Returns:

The learning rate, this is value is essentially a percent. It is the degree to which the gradients are applied to the weight matrix to allow learning.

isValidResume

public boolean isValidResume(TrainingContinuation state)

Determine if the specified continuation object is valid to resume with.

Parameters:

state - The continuation object to check.

Returns:

True if the specified continuation object is valid for this training method and network.

pause

public TrainingContinuation pause()

Pause the training.

Specified by:

pause in interface MLTrain

Returns:

A training continuation object to continue with.

resume

public void resume(TrainingContinuation state)

Resume training.

Specified by:

resume in interface MLTrain

Parameters:

state - The training state to return to.

setLearningRate

public void setLearningRate(double rate)

Set the learning rate, this is value is essentially a percent. It is the degree to which the gradients are applied to the weight matrix to allow learning.

Specified by:

setLearningRate in interface LearningRate

Parameters:

rate - The learning rate.

getOutputEpsilon

public double getOutputEpsilon()

Returns:

the outputEpsilon

getShrink

public double getShrink()

Returns:

the shrink

setShrink

public void setShrink(double s)

Parameters:

s - the shrink to set

setOutputEpsilon

public void setOutputEpsilon(double theOutputEpsilon)

Parameters:

theOutputEpsilon - the outputEpsilon to set

initOthers

public void initOthers()

Perform training method specific init.

Specified by:

initOthers in class Propagation

updateWeight

public double updateWeight(double[] gradients,
                           double[] lastGradient,
                           int index)

Update a weight.

Specified by:

updateWeight in class Propagation

Parameters:

gradients - The gradients.

lastGradient - The last gradients.

index - The index.

Returns:

The weight delta.

updateWeight

public double updateWeight(double[] gradients,
                           double[] lastGradient,
                           int index,
                           double dropoutRate)

Update a weight with droput.

Specified by:

updateWeight in class Propagation

Parameters:

gradients - The gradients.

lastGradient - The last gradients.

index - The index.

dropoutRate - The dropout rate.

Returns:

The weight delta.

setBatchSize

public void setBatchSize(int theBatchSize)

Do not allow batch sizes other than 0, not supported.

Specified by:

setBatchSize in interface BatchSize

Overrides:

setBatchSize in class Propagation

Parameters:

theBatchSize - The batch size.