jsat.math.optimization.stochastic

Interface GradientUpdater

All Superinterfaces:

java.io.Serializable

All Known Implementing Classes:

AdaDelta, AdaGrad, Adam, NAdaGrad, RMSProp, Rprop, SGDMomentum, SimpleSGD

public interface GradientUpdater
extends java.io.Serializable

This interface defines the method of updating some weight vector using a gradient and a learning rate. The method may then apply its own set of learning rates on top of the given learning rate in order to accelerate convergence in general or for specific conditions / methods.

Method Summary

Modifier and Type	Method and Description
GradientUpdater	clone()
void	setup(int d) Sets up this updater to update a weight vector of dimension d by a gradient of the same dimension
void	update(Vec w, Vec grad, double eta) Updates the weight vector x such that x = x-ηf(grad), where f(grad) is some function on the gradient that effectively returns a new vector.
double	update(Vec w, Vec grad, double eta, double bias, double biasGrad) Updates the weight vector x such that x = x-ηf(grad), where f(grad) is some function on the gradient that effectively returns a new vector.

Method Detail

update

void update(Vec w,
            Vec grad,
            double eta)

Updates the weight vector x such that x = x-ηf(grad), where f(grad) is some function on the gradient that effectively returns a new vector. It is not necessary for the internal implementation to ever explicitly form any of these objects, so long as x is mutated to have the correct result.

Parameters:

w - the vector to mutate such that is has been updated by the gradient

grad - the gradient to update the weight vector x from

eta - the learning rate to apply

update

double update(Vec w,
              Vec grad,
              double eta,
              double bias,
              double biasGrad)

Updates the weight vector x such that x = x-ηf(grad), where f(grad) is some function on the gradient that effectively returns a new vector. It is not necessary for the internal implementation to ever explicitly form any of these objects, so long as x is mutated to have the correct result.

This version of the update method includes two extra parameters to make it easer to use when a scalar bias term is also used

Parameters:

w - the vector to mutate such that is has been updated by the gradient

grad - the gradient to update the weight vector x from

eta - the learning rate to apply

bias - the bias term of the vector

biasGrad - the gradient for the bias term

Returns:

the value to change the bias by, the update being bias = bias - returnValue

setup

void setup(int d)

Sets up this updater to update a weight vector of dimension d by a gradient of the same dimension

Parameters:

d - the dimension of the weight vector that will be updated

clone

GradientUpdater clone()