jsat.math.optimization

Class ModifiedOWLQN

java.lang.Object

jsat.math.optimization.ModifiedOWLQN

All Implemented Interfaces:

Optimizer

public class ModifiedOWLQN
extends java.lang.Object
implements Optimizer

This implements the Modified Orthant-Wise Limited memory Quasi-Newton(mOWL-QN) optimizer. This algorithm is an extension of LBFGS, and solves minimization problems of the form: f(x) + λ ||x||₁. It requires the function and it's gradient to work.

See:

• Gong, P., & Ye, J. (2015). A Modified Orthant-Wise Limited Memory Quasi-Newton Method with Convergence Analysis. In The 32nd International Conference on Machine Learning (Vol. 37).
• Andrew, G., & Gao, J. (2007). Scalable training of L1 -regularized log-linear models. In Proceedings of the 24th international conference on Machine learning - ICML ’07 (pp. 33–40). New York, New York, USA: ACM Press. doi:10.1145/1273496.1273501

Constructor Summary

Constructors

Constructor and Description
ModifiedOWLQN() Creates a new mOWL-QN optimizer with no regularization penalty
ModifiedOWLQN(double lambda) Creates a new mOWL-QN optimizer

Method Summary

Modifier and Type	Method and Description
ModifiedOWLQN	clone()
double	getBeta()
double	getEps()
Vec	getLambdaMultipler()
int	getM() Returns the number of history items that will be used
int	getMaximumIterations() Returns the maximum number of iterations to perform
void	optimize(double tolerance, Vec w, Vec x0, Function f, FunctionVec fp, boolean parallel) Attempts to optimize the given function by finding the value of w that will minimize the value returned by f(w), using w = x0 as an initial starting point.
void	setBeta(double beta) Sets the shrinkage term used for the line search.
void	setEps(double eps) Sets the epsilon term that helps control when the gradient descent step is taken instead of the normal Quasi-Newton step.
void	setLambda(double lambda) Sets the regularization term for the optimizer
void	setLambdaMultipler(Vec lambdaMultipler) This method sets a vector that will contain a separate multiplier for lambda for each dimension of the problem.
void	setM(int m) Sets the number of history items to keep that are used to approximate the Hessian of the problem
void	setMaximumIterations(int iterations) Sets the maximum number of iterations allowed for the optimization method

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface jsat.math.optimization.Optimizer

optimize

Constructor Detail

ModifiedOWLQN

public ModifiedOWLQN()

Creates a new mOWL-QN optimizer with no regularization penalty

ModifiedOWLQN

public ModifiedOWLQN(double lambda)

Creates a new mOWL-QN optimizer

Parameters:

lambda - the regularization penalty to use

Method Detail

setLambda

public void setLambda(double lambda)

Sets the regularization term for the optimizer

Parameters:

lambda - the regularization penalty

setLambdaMultipler

public void setLambdaMultipler(Vec lambdaMultipler)

This method sets a vector that will contain a separate multiplier for lambda for each dimension of the problem. This allows for each dimension to have a different regularization penalty.

If set to null, all dimensions will simply use λ as their regularization value.

Parameters:

lambdaMultipler - the per-dimension regularization multiplier, or null.

getLambdaMultipler

public Vec getLambdaMultipler()

setM

public void setM(int m)

Sets the number of history items to keep that are used to approximate the Hessian of the problem

Parameters:

m - the number of history items to keep

getM

public int getM()

Returns the number of history items that will be used

Returns:

the number of history items that will be used

setEps

public void setEps(double eps)

Sets the epsilon term that helps control when the gradient descent step is taken instead of the normal Quasi-Newton step. Larger values cause more GD steps. You shouldn't need to alter this variable

Parameters:

eps - tolerance term for GD steps

getEps

public double getEps()

setBeta

public void setBeta(double beta)

Sets the shrinkage term used for the line search.

Parameters:

beta - the line search shrinkage term

getBeta

public double getBeta()

optimize

public void optimize(double tolerance,
                     Vec w,
                     Vec x0,
                     Function f,
                     FunctionVec fp,
                     boolean parallel)

Description copied from interface: Optimizer

Attempts to optimize the given function by finding the value of w that will minimize the value returned by f(w), using w = x₀ as an initial starting point.

Specified by:

optimize in interface Optimizer

Parameters:

tolerance - the value that the gradient norm must be less than to consider converged

w - the the location to store the final solution

x0 - the initial guess for the solution. This value will not be changed, and intermediate matrices will be created as the same type.

f - the objective function to minimizer

fp - the derivative of the objective function, may be null depending on the optimizer

parallel - true if multiple threads should be used for optimization, or false if a single thread should be used.

setMaximumIterations

public void setMaximumIterations(int iterations)

Description copied from interface: Optimizer

Sets the maximum number of iterations allowed for the optimization method

Specified by:

setMaximumIterations in interface Optimizer

Parameters:

iterations - the maximum number of iterations to perform

getMaximumIterations

public int getMaximumIterations()

Description copied from interface: Optimizer

Returns the maximum number of iterations to perform

Specified by:

getMaximumIterations in interface Optimizer

Returns:

the maximum number of iterations to perform

clone

public ModifiedOWLQN clone()

Specified by:

clone in interface Optimizer

Overrides:

clone in class java.lang.Object