org.joone.engine.extenders

Class WeightDecayExtender

java.lang.Object

org.joone.engine.extenders.LearnerExtender

org.joone.engine.extenders.DeltaRuleExtender

org.joone.engine.extenders.WeightDecayExtender

public class WeightDecayExtender
extends DeltaRuleExtender

Weight decay adds a penalty term to the error function. The penalty term penalizes large weights. The weight decay penalty term causes the weights to converge to smaller absolute values than they otherwise would. Smaller weights are expected to improve generalization. The update formula is changed in: Dw(t+1) = dw(t+1) - d x w(t) d is a weight decay value.

Constructor Summary

Constructors

Constructor and Description
WeightDecayExtender() Creates a new instance of WeightDecayExtender

Method Summary

Modifier and Type	Method and Description
double	getDecay() Gets the decay parameter.
double	getDelta(double[] currentGradientOuts, int j, double aPreviousDelta) Computes the delta value for a bias.
double	getDelta(double[] currentInps, int j, double[] currentPattern, int k, double aPreviousDelta) Computes the delta value for a weight.
void	postBiasUpdate(double[] currentGradientOuts) Gives extenders a change to do some post-computing after the biases are updated.
void	postWeightUpdate(double[] currentPattern, double[] currentInps) Gives extenders a change to do some post-computing after the weights are updated.
void	preBiasUpdate(double[] currentGradientOuts) Gives extenders a change to do some pre-computing before the biases are updated.
void	preWeightUpdate(double[] currentPattern, double[] currentInps) Gives extenders a change to do some pre-computing before the weights are updated.
void	setDecay(double aDecay) Sets the decay parameter.

Methods inherited from class org.joone.engine.extenders.LearnerExtender

isEnabled, setEnabled, setLearner

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WeightDecayExtender

public WeightDecayExtender()

Creates a new instance of WeightDecayExtender

Method Detail

getDelta

public double getDelta(double[] currentGradientOuts,
                       int j,
                       double aPreviousDelta)

Description copied from class: DeltaRuleExtender

Computes the delta value for a bias.

Specified by:

getDelta in class DeltaRuleExtender

Parameters:

currentGradientOuts - the back propagated gradients.

j - the index of the bias.

aPreviousDelta - a delta value calculated by a previous delta extender.

getDelta

public double getDelta(double[] currentInps,
                       int j,
                       double[] currentPattern,
                       int k,
                       double aPreviousDelta)

Description copied from class: DeltaRuleExtender

Computes the delta value for a weight.

Specified by:

getDelta in class DeltaRuleExtender

Parameters:

currentInps - the forwarded input.

j - the input index of the weight.

currentPattern - the back propagated gradients.

k - the output index of the weight.

aPreviousDelta - a delta value calculated by a previous delta extender.

postBiasUpdate

public void postBiasUpdate(double[] currentGradientOuts)

Description copied from class: LearnerExtender

Gives extenders a change to do some post-computing after the biases are updated.

Specified by:

postBiasUpdate in class LearnerExtender

Parameters:

currentGradientOuts - the back propagated gradients.

postWeightUpdate

public void postWeightUpdate(double[] currentPattern,
                             double[] currentInps)

Description copied from class: LearnerExtender

Gives extenders a change to do some post-computing after the weights are updated.

Specified by:

postWeightUpdate in class LearnerExtender

Parameters:

currentPattern - the back propagated gradients.

currentInps - the forwarded input.

preBiasUpdate

public void preBiasUpdate(double[] currentGradientOuts)

Description copied from class: LearnerExtender

Gives extenders a change to do some pre-computing before the biases are updated.

Specified by:

preBiasUpdate in class LearnerExtender

Parameters:

currentGradientOuts - the back propagated gradients.

preWeightUpdate

public void preWeightUpdate(double[] currentPattern,
                            double[] currentInps)

Description copied from class: LearnerExtender

Gives extenders a change to do some pre-computing before the weights are updated.

Specified by:

preWeightUpdate in class LearnerExtender

Parameters:

currentPattern - the back propagated gradients.

currentInps - the forwarded input.

setDecay

public void setDecay(double aDecay)

Sets the decay parameter.

Parameters:

aDecay - the decay parameter value.

getDecay

public double getDecay()

Gets the decay parameter.

Returns:

the decay parameter.