jsat.classifiers.linear

Class SCD

java.lang.Object

jsat.classifiers.linear.SCD

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, Classifier, Parameterized, Regressor, SimpleWeightVectorModel, SingleWeightVectorModel

public class SCD
extends java.lang.Object
implements Classifier, Regressor, Parameterized, SingleWeightVectorModel

Implementation of Stochastic Coordinate Descent for L1 regularized classification and regression. Which one is supported is controlled by the LossFunc used. To be used the loss function must be twice differentiable with a finite maximal second derivative value. LogisticLoss for classification and SquaredLoss for regression are the ones used in the original paper.

Because the SCD needs column major data for efficient implementation, a second copy of data will be created in memory during training.

See: Shalev-Shwartz, S.,&Tewari, A. (2009). Stochastic Methods for L₁-regularized Loss Minimization. In 26th International Conference on Machine Learning (Vol. 12, pp. 929–936). Retrieved from here

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SCD(LossFunc loss, double regularization, int iterations) Creates anew SCD learner
SCD(SCD toCopy) Copy constructor

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
SCD	clone()
double	getBias() Returns the bias term used for the model, or 0 of the model does not support or was not trained with a bias term.
double	getBias(int index) Returns the bias term used with the weight vector for the given class index.
int	getIterations() Returns the number of iterations used
Vec	getRawWeight() Returns the only weight vector used for the model
Vec	getRawWeight(int index) Returns the raw weight vector associated with the given class index.
double	getRegularization() Returns the regularization parameter value used for learning.
int	numWeightsVecs() Returns the number of weight vectors that can be returned.
double	regress(DataPoint data)
void	setIterations(int iterations) Sets the number of iterations that will be used.
void	setRegularization(double regularization) Sets the regularization constant used for learning.
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	train(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	train(ClassificationDataSet dataSet, boolean parallel) Trains the classifier and constructs a model for classification using the given data set.
void	train(RegressionDataSet dataSet)
void	train(RegressionDataSet dataSet, boolean parallel)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface jsat.parameters.Parameterized

getParameter, getParameters

Constructor Detail

SCD

public SCD(LossFunc loss,
           double regularization,
           int iterations)

Creates anew SCD learner

Parameters:

loss - the loss function to use

regularization - the regularization term to used

iterations - the number of iterations to perform

SCD

public SCD(SCD toCopy)

Copy constructor

Parameters:

toCopy - the object to copy

Method Detail

setIterations

public void setIterations(int iterations)

Sets the number of iterations that will be used.

Parameters:

iterations - the number of training iterations

getIterations

public int getIterations()

Returns the number of iterations used

Returns:

the number of iterations used

setRegularization

public void setRegularization(double regularization)

Sets the regularization constant used for learning. The regularization must be positive, and the learning rate is proportional to the regularization value. This means regularizations very near zero will take a long time to converge.

Parameters:

regularization - the regularization to apply in (0, Infinity)

getRegularization

public double getRegularization()

Returns the regularization parameter value used for learning.

Returns:

the regularization parameter value used for learning.

getRawWeight

public Vec getRawWeight()

Description copied from interface: SingleWeightVectorModel

Returns the only weight vector used for the model

Specified by:

getRawWeight in interface SingleWeightVectorModel

Returns:

the only weight vector used for the model

getBias

public double getBias()

Description copied from interface: SingleWeightVectorModel

Returns the bias term used for the model, or 0 of the model does not support or was not trained with a bias term.

Specified by:

getBias in interface SingleWeightVectorModel

Returns:

the bias term for the model

getRawWeight

public Vec getRawWeight(int index)

Description copied from interface: SimpleWeightVectorModel

Returns the raw weight vector associated with the given class index. If the given class is an implicit zero vector, a ConstantVector object may be returned.
Do not alter the returned weight vector, as it will change the model's values.

If a regression problem, only index = 0 should be used

Specified by:

getRawWeight in interface SimpleWeightVectorModel

Parameters:

index - the class index to get the weight vector for

Returns:

the weight vector used for the specified class

getBias

public double getBias(int index)

Description copied from interface: SimpleWeightVectorModel

Returns the bias term used with the weight vector for the given class index. If the model does not support or was not trained with bias weights, 0 will be returned.

If a regression problem, only index = 0 should be used

Specified by:

getBias in interface SimpleWeightVectorModel

Parameters:

index - the class index to get the weight vector for

Returns:

the bias term for the specified class

numWeightsVecs

public int numWeightsVecs()

Description copied from interface: SimpleWeightVectorModel

Returns the number of weight vectors that can be returned. For binary classification problems the value may be 1 if only a single weight vector's sign is used to determine the class. For multi-class problems, the weight vector count includes the implicit zero vector (if one is being used).

Specified by:

numWeightsVecs in interface SimpleWeightVectorModel

Returns:

the number of weight vectors for which SimpleWeightVectorModel.getRawWeight(int) can be called.

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Specified by:

classify in interface Classifier

Parameters:

data - the data point to classify

Returns:

the results of the classification.

train

public void train(ClassificationDataSet dataSet,
                  boolean parallel)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Specified by:

train in interface Classifier

Parameters:

dataSet - the data set to train on

parallel - true if multiple threads should be used to train the model. false if it should be done in a single threaded manner.

train

public void train(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Specified by:

train in interface Classifier

Parameters:

dataSet - the data set to train on

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Specified by:

supportsWeightedData in interface Classifier

Specified by:

supportsWeightedData in interface Regressor

Returns:

true if the model supports weighted data, false otherwise

regress

public double regress(DataPoint data)

Specified by:

regress in interface Regressor

train

public void train(RegressionDataSet dataSet,
                  boolean parallel)

Specified by:

train in interface Regressor

train

public void train(RegressionDataSet dataSet)

Specified by:

train in interface Regressor

clone

public SCD clone()

Specified by:

clone in interface Classifier

Specified by:

clone in interface Regressor

Overrides:

clone in class java.lang.Object