jsat.classifiers.linear

Class SDCA

java.lang.Object

jsat.classifiers.linear.SDCA

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, Classifier, WarmClassifier, Parameterized, Regressor, WarmRegressor, SimpleWeightVectorModel

public class SDCA
extends java.lang.Object
implements Classifier, Regressor, Parameterized, SimpleWeightVectorModel, WarmClassifier, WarmRegressor

This class implements the Proximal Stochastic Dual Coordinate Ascent (SDCA) algorithm for learning general linear models with Elastic-Net regularization. It is a fast learning algorithm, and can be used for generic Logistic or least-squares regression with elastic-net regularization.
It can work with any LossFunc to determine if it solves a classification or regression problem, so long as the conjugate method of the loss is implemented properly. This is especially useful for more exotic cases, like using the robust Huber loss in a L1 regularized scenario.
NOTE: The current implementation dose not support any multi-class loss function, though that isn't a limitation of the algorithm.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SDCA() Creates a new SDCA learner for logistic-regression.
SDCA(double lambda) The implementation will use Elastic-Net regularization by default.
SDCA(double lambda, LossFunc loss) Creates a new SDCA learner for either a classification or regression problem, the type of which is determined by the LossFunction given.
SDCA(SDCA toCopy) Copy Constructor

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
SDCA	clone()
double	getAlpha()
double	getBias(int index) Returns the bias term used with the weight vector for the given class index.
double	getLambda()
LossFunc	getLoss() Returns the loss function in use
int	getMaxIters()
Vec	getRawWeight(int index) Returns the raw weight vector associated with the given class index.
double	getTolerance()
static Distribution	guessAlpha(DataSet d) Guess the distribution to use for the regularization balance alpha.
static Distribution	guessLambda(DataSet d) Guess the distribution to use for the regularization term lambda.
boolean	isUseBias() Returns whether or not an implicit bias term is in use
int	numWeightsVecs() Returns the number of weight vectors that can be returned.
double	regress(DataPoint data)
void	setAlpha(double alpha) Using α = 1 corresponds to pure L1 regularization, and α = 0 corresponds to pure L2 regularization.
void	setLambda(double lambda) Sets the regularization term, where larger values indicate a larger regularization penalty.
void	setLoss(LossFunc loss) Sets the loss function used for the model.
void	setMaxIters(int maxOuterIters) Sets the maximum number of training iterations (epochs) for the algorithm.
void	setTolerance(double e_out) Sets the tolerance parameter for convergence.
void	setUseBias(boolean useBias) Sets whether or not an implicit bias term will be added to the data set
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	train(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	train(ClassificationDataSet dataSet, boolean parallel) Trains the classifier and constructs a model for classification using the given data set.
void	train(ClassificationDataSet dataSet, Classifier warmSolution) Trains the classifier and constructs a model for classification using the given data set.
void	train(ClassificationDataSet dataSet, Classifier warmSolution, boolean parallel) Trains the classifier and constructs a model for classification using the given data set.
void	train(RegressionDataSet dataSet)
void	train(RegressionDataSet dataSet, boolean parallel)
void	train(RegressionDataSet dataSet, Regressor warmSolution) Trains the regressor and constructs a model for regression using the given data set.
void	train(RegressionDataSet dataSet, Regressor warmSolution, boolean parallel) Trains the regressor and constructs a model for regression using the given data set.
boolean	warmFromSameDataOnly() Some models can only be warm started from a solution trained on the exact same data set as the model it is warm starting from.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface jsat.parameters.Parameterized

getParameter, getParameters

Constructor Detail

SDCA

public SDCA()

Creates a new SDCA learner for logistic-regression. Pure L2 or L1 regularization can be obtained using the alpha parameter.

SDCA

public SDCA(double lambda)

The implementation will use Elastic-Net regularization by default. Pure L2 or L1 regularization can be obtained using the alpha parameter.

Parameters:

lambda - the regularization penalty to use.

SDCA

public SDCA(double lambda,
            LossFunc loss)

Creates a new SDCA learner for either a classification or regression problem, the type of which is determined by the LossFunction given.
The implementation will use Elastic-Net regularization by default. Pure L2 or L1 regularization can be obtained using the alpha parameter.

Parameters:

lambda - the regularization penalty to use.

loss - the loss function to use for training, which determines if it implements classification or regression

SDCA

public SDCA(SDCA toCopy)

Copy Constructor

Parameters:

toCopy - the object to copy

Method Detail

setUseBias

public void setUseBias(boolean useBias)

Sets whether or not an implicit bias term will be added to the data set

Parameters:

useBias - true to add an implicit bias term

isUseBias

public boolean isUseBias()

Returns whether or not an implicit bias term is in use

Returns:

true if a bias term is in use

setLambda

public void setLambda(double lambda)

Sets the regularization term, where larger values indicate a larger regularization penalty.

Parameters:

lambda - the positive regularization term

getLambda

public double getLambda()

Returns:

the regularization term

setAlpha

public void setAlpha(double alpha)

Using α = 1 corresponds to pure L₁ regularization, and α = 0 corresponds to pure L₂ regularization. Any value in-between is then an Elastic Net regularization.

Parameters:

alpha - the value in [0, 1] for determining the regularization penalty's interpolation between pure L₂ and L₁ regularization.

getAlpha

public double getAlpha()

Returns:

the fraction of weight (in [0, 1]) to apply to L₁ regularization instead of L₂ regularization.

setMaxIters

public void setMaxIters(int maxOuterIters)

Sets the maximum number of training iterations (epochs) for the algorithm.

Parameters:

maxOuterIters - the maximum number of outer iterations

getMaxIters

public int getMaxIters()

Returns:

the maximum number of training iterations

setTolerance

public void setTolerance(double e_out)

Sets the tolerance parameter for convergence. Smaller values will be more exact, but larger values will converge faster. The default value is fairly exact at , increasing it by an order of magnitude can often be done without hurting accuracy.

Parameters:

e_out - the tolerance parameter.

getTolerance

public double getTolerance()

Returns:

the convergence tolerance parameter

setLoss

public void setLoss(LossFunc loss)

Sets the loss function used for the model. The loss function controls whether or not regression, binary classification, or multi-class classification is supported.
NOTE: SDCA requires that the given loss function implement the conjugate method, otherwise it will not work with this algorithm.

Parameters:

loss - the loss function to use

getLoss

public LossFunc getLoss()

Returns the loss function in use

Returns:

the loss function in use

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Specified by:

classify in interface Classifier

Parameters:

data - the data point to classify

Returns:

the results of the classification.

regress

public double regress(DataPoint data)

Specified by:

regress in interface Regressor

train

public void train(ClassificationDataSet dataSet,
                  boolean parallel)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Specified by:

train in interface Classifier

Parameters:

dataSet - the data set to train on

parallel - true if multiple threads should be used to train the model. false if it should be done in a single threaded manner.

train

public void train(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Specified by:

train in interface Classifier

Parameters:

dataSet - the data set to train on

train

public void train(ClassificationDataSet dataSet,
                  Classifier warmSolution,
                  boolean parallel)

Description copied from interface: WarmClassifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Specified by:

train in interface WarmClassifier

Parameters:

dataSet - the data set to train on

warmSolution - the solution to use to warm start this model

parallel - true if the training should be done using multiple-cores, false for single threaded.

train

public void train(ClassificationDataSet dataSet,
                  Classifier warmSolution)

Description copied from interface: WarmClassifier

Trains the classifier and constructs a model for classification using the given data set.

Specified by:

train in interface WarmClassifier

Parameters:

dataSet - the data set to train on

warmSolution - the solution to use to warm start this model

train

public void train(RegressionDataSet dataSet,
                  boolean parallel)

Specified by:

train in interface Regressor

train

public void train(RegressionDataSet dataSet)

Specified by:

train in interface Regressor

train

public void train(RegressionDataSet dataSet,
                  Regressor warmSolution,
                  boolean parallel)

Description copied from interface: WarmRegressor

Trains the regressor and constructs a model for regression using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Specified by:

train in interface WarmRegressor

Parameters:

dataSet - the data set to train on

warmSolution - the solution to use to warm start this model

parallel - true if the training should be done using multiple-cores, false for single threaded.

train

public void train(RegressionDataSet dataSet,
                  Regressor warmSolution)

Description copied from interface: WarmRegressor

Trains the regressor and constructs a model for regression using the given data set.

Specified by:

train in interface WarmRegressor

Parameters:

dataSet - the data set to train on

warmSolution - the solution to use to warm start this model

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Specified by:

supportsWeightedData in interface Classifier

Specified by:

supportsWeightedData in interface Regressor

Returns:

true if the model supports weighted data, false otherwise

clone

public SDCA clone()

Specified by:

clone in interface Classifier

Specified by:

clone in interface Regressor

Overrides:

clone in class java.lang.Object

getRawWeight

public Vec getRawWeight(int index)

Description copied from interface: SimpleWeightVectorModel

Returns the raw weight vector associated with the given class index. If the given class is an implicit zero vector, a ConstantVector object may be returned.
Do not alter the returned weight vector, as it will change the model's values.

If a regression problem, only index = 0 should be used

Specified by:

getRawWeight in interface SimpleWeightVectorModel

Parameters:

index - the class index to get the weight vector for

Returns:

the weight vector used for the specified class

getBias

public double getBias(int index)

Description copied from interface: SimpleWeightVectorModel

Returns the bias term used with the weight vector for the given class index. If the model does not support or was not trained with bias weights, 0 will be returned.

If a regression problem, only index = 0 should be used

Specified by:

getBias in interface SimpleWeightVectorModel

Parameters:

index - the class index to get the weight vector for

Returns:

the bias term for the specified class

numWeightsVecs

public int numWeightsVecs()

Description copied from interface: SimpleWeightVectorModel

Returns the number of weight vectors that can be returned. For binary classification problems the value may be 1 if only a single weight vector's sign is used to determine the class. For multi-class problems, the weight vector count includes the implicit zero vector (if one is being used).

Specified by:

numWeightsVecs in interface SimpleWeightVectorModel

Returns:

the number of weight vectors for which SimpleWeightVectorModel.getRawWeight(int) can be called.

warmFromSameDataOnly

public boolean warmFromSameDataOnly()

Description copied from interface: WarmClassifier

Some models can only be warm started from a solution trained on the exact same data set as the model it is warm starting from. If this is the case true will be returned. The behavior for training on a different data set when this is defined is undefined. It may cause an error, or it may cause the algorithm to take longer or reach a worse solution.
When true, it is important that the data set be unaltered - this includes mutating the values stored or re-arranging the data points within the data set.

Specified by:

warmFromSameDataOnly in interface WarmClassifier

Specified by:

warmFromSameDataOnly in interface WarmRegressor

Returns:

true if the algorithm can only be warm started from the model trained on the exact same data set.

guessLambda

public static Distribution guessLambda(DataSet d)

Guess the distribution to use for the regularization term lambda.

Parameters:

d - the data set to get the guess for

Returns:

the guess for the lambda parameter

guessAlpha

public static Distribution guessAlpha(DataSet d)

Guess the distribution to use for the regularization balance alpha.

Parameters:

d - the data set to get the guess for

Returns:

the guess for the lambda parameter