jsat.classifiers.neuralnetwork

Class SGDNetworkTrainer

java.lang.Object

jsat.classifiers.neuralnetwork.SGDNetworkTrainer

All Implemented Interfaces:

java.io.Serializable

public class SGDNetworkTrainer
extends java.lang.Object
implements java.io.Serializable

This class provides a highly configurable and generalized method of training a neural network using Stochastic Gradient Decent.

Note, the API of this class may change in the future.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SGDNetworkTrainer() Creates a new SGD network training that uses dropout
SGDNetworkTrainer(SGDNetworkTrainer toCopy) Copy constructor

Method Summary

Modifier and Type	Method and Description
Vec	feedfoward(Vec x) Feeds the given singular pattern through the network and computes its activations
void	finishUpdating() Calling this method indicates that the user has no intentions of updating the network again and is ready to use it for prediction.
BiastInitializer	getBiasInit()
double	getDropoutHidden()
double	getDropoutInput()
double	getEta()
DecayRate	getEtaDecay()
GradientUpdater	getGradientUpdater()
int[]	getLayerSizes()
WeightRegularizer	getRegularizer()
WeightInitializer	getWeightInit()
void	setBiasInit(BiastInitializer biasInit) Sets the method to use when initializing neuron bias values
void	setDropoutHidden(double p) Sets the probability of dropping a value from the hidden layer
void	setDropoutInput(double p) Sets the probability of dropping a value from the input layer
void	setEta(double eta) Sets the base global learning rate.
void	setEtaDecay(DecayRate etaDecay) Sets the decay rate on the global learning rate over time
void	setGradientUpdater(GradientUpdater updater) Sets the gradient update that will be used when updating the weight matrices and bias terms.
void	setLayersActivation(java.util.List<ActivationLayer> layersActivation) Sets the list of layer activations for all layers other than the input layer.
void	setLayerSizes(int... layerSizes) Sets the array indicating the total number of layers in the network and the sizes of each layer.
void	setRegularizer(WeightRegularizer regularizer) Sets the method of regularizing the connections weights
void	setup() Prepares the network by creating all needed structure, initializing weights, and preparing it for updates
void	setWeightInit(WeightInitializer weightInit) Sets the method used to initialize matrix connection weights
double	updateMiniBatch(java.util.List<Vec> x, java.util.List<Vec> y) Performs a mini-batch update of the network using the given input and output pairs
double	updateMiniBatch(java.util.List<Vec> x, java.util.List<Vec> y, java.util.concurrent.ExecutorService ex) Performs a mini-batch update of the network using the given input and output pairs

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SGDNetworkTrainer

public SGDNetworkTrainer()

Creates a new SGD network training that uses dropout

SGDNetworkTrainer

public SGDNetworkTrainer(SGDNetworkTrainer toCopy)

Copy constructor

Parameters:

toCopy - the object to copy

Method Detail

setDropoutInput

public void setDropoutInput(double p)

Sets the probability of dropping a value from the input layer

Parameters:

p - the probability in [0, 1) of dropping a value in the input layer

getDropoutInput

public double getDropoutInput()

Returns:

the dropout probability for the input layer

setDropoutHidden

public void setDropoutHidden(double p)

Sets the probability of dropping a value from the hidden layer

Parameters:

p - the probability in [0, 1) of dropping a value in the hidden layer

getDropoutHidden

public double getDropoutHidden()

Returns:

the dropout probability for the hidden layers

setEtaDecay

public void setEtaDecay(DecayRate etaDecay)

Sets the decay rate on the global learning rate over time

Parameters:

etaDecay - the decay rate to use

getEtaDecay

public DecayRate getEtaDecay()

Returns:

the decay rate in use

setEta

public void setEta(double eta)

Sets the base global learning rate.

Parameters:

eta - the learning rate to use

getEta

public double getEta()

Returns:

the global learning rate used

setRegularizer

public void setRegularizer(WeightRegularizer regularizer)

Sets the method of regularizing the connections weights

Parameters:

regularizer - the method of regularizing the network

getRegularizer

public WeightRegularizer getRegularizer()

Returns:

the regularizer for the network

setLayerSizes

public void setLayerSizes(int... layerSizes)

Sets the array indicating the total number of layers in the network and the sizes of each layer. The length of the array is the number of layers and the value at each index is the size of that layer.

Parameters:

layerSizes - the array of layer sizes

getLayerSizes

public int[] getLayerSizes()

Returns:

the array of layer sizes in the network

setLayersActivation

public void setLayersActivation(java.util.List<ActivationLayer> layersActivation)

Sets the list of layer activations for all layers other than the input layer.

Parameters:

layersActivation - the list of hidden and output layer activations

setGradientUpdater

public void setGradientUpdater(GradientUpdater updater)

Sets the gradient update that will be used when updating the weight matrices and bias terms.

Parameters:

updater - the updater to use

getGradientUpdater

public GradientUpdater getGradientUpdater()

Returns:

the gradient updater used

setWeightInit

public void setWeightInit(WeightInitializer weightInit)

Sets the method used to initialize matrix connection weights

Parameters:

weightInit - the weight initialization method

getWeightInit

public WeightInitializer getWeightInit()

Returns:

the weight initialization method

setBiasInit

public void setBiasInit(BiastInitializer biasInit)

Sets the method to use when initializing neuron bias values

Parameters:

biasInit - the bias initialization method

getBiasInit

public BiastInitializer getBiasInit()

Returns:

the bias initialization method

setup

public void setup()

Prepares the network by creating all needed structure, initializing weights, and preparing it for updates

finishUpdating

public void finishUpdating()

Calling this method indicates that the user has no intentions of updating the network again and is ready to use it for prediction. This will remove objects not needed for prediction and do cleanup.

updateMiniBatch

public double updateMiniBatch(java.util.List<Vec> x,
                              java.util.List<Vec> y)

Performs a mini-batch update of the network using the given input and output pairs

Parameters:

x - the list of input values

y - the list of output values

Returns:

the error incurred on the given mini batch

updateMiniBatch

public double updateMiniBatch(java.util.List<Vec> x,
                              java.util.List<Vec> y,
                              java.util.concurrent.ExecutorService ex)

Performs a mini-batch update of the network using the given input and output pairs

Parameters:

x - the list of input values

y - the list of output values

ex - the source of threads for parallel computation, may be null

Returns:

the error incurred on the given mini batch

feedfoward

public Vec feedfoward(Vec x)

Feeds the given singular pattern through the network and computes its activations

Parameters:

x - the input vector to feed forward through the network

Returns:

the final activation for this network