jsat.math.optimization.stochastic

Class SimpleSGD

java.lang.Object

jsat.math.optimization.stochastic.SimpleSGD

All Implemented Interfaces:

java.io.Serializable, GradientUpdater

public class SimpleSGD
extends java.lang.Object
implements GradientUpdater

Performs unaltered Stochastic Gradient Decent updates computing x = x- η grad

Because the SimpleSGD requires no internal state, it is not necessary to call setup(int).

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SimpleSGD() Creates a new SGD updater

Method Summary

Modifier and Type	Method and Description
SimpleSGD	clone()
void	setup(int d) Sets up this updater to update a weight vector of dimension d by a gradient of the same dimension
void	update(Vec x, Vec grad, double eta) Updates the weight vector x such that x = x-ηf(grad), where f(grad) is some function on the gradient that effectively returns a new vector.
double	update(Vec x, Vec grad, double eta, double bias, double biasGrad) Updates the weight vector x such that x = x-ηf(grad), where f(grad) is some function on the gradient that effectively returns a new vector.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SimpleSGD

public SimpleSGD()

Creates a new SGD updater

Method Detail

update

public void update(Vec x,
                   Vec grad,
                   double eta)

Description copied from interface: GradientUpdater

Updates the weight vector x such that x = x-ηf(grad), where f(grad) is some function on the gradient that effectively returns a new vector. It is not necessary for the internal implementation to ever explicitly form any of these objects, so long as x is mutated to have the correct result.

Specified by:

update in interface GradientUpdater

Parameters:

x - the vector to mutate such that is has been updated by the gradient

grad - the gradient to update the weight vector x from

eta - the learning rate to apply

update

public double update(Vec x,
                     Vec grad,
                     double eta,
                     double bias,
                     double biasGrad)

Description copied from interface: GradientUpdater

Updates the weight vector x such that x = x-ηf(grad), where f(grad) is some function on the gradient that effectively returns a new vector. It is not necessary for the internal implementation to ever explicitly form any of these objects, so long as x is mutated to have the correct result.

This version of the update method includes two extra parameters to make it easer to use when a scalar bias term is also used

Specified by:

update in interface GradientUpdater

Parameters:

x - the vector to mutate such that is has been updated by the gradient

grad - the gradient to update the weight vector x from

eta - the learning rate to apply

bias - the bias term of the vector

biasGrad - the gradient for the bias term

Returns:

the value to change the bias by, the update being bias = bias - returnValue

clone

public SimpleSGD clone()

Specified by:

clone in interface GradientUpdater

Overrides:

clone in class java.lang.Object

setup

public void setup(int d)

Description copied from interface: GradientUpdater

Sets up this updater to update a weight vector of dimension d by a gradient of the same dimension

Specified by:

setup in interface GradientUpdater

Parameters:

d - the dimension of the weight vector that will be updated