jsat.classifiers.linear

Class LinearSGD

java.lang.Object

jsat.classifiers.BaseUpdateableClassifier

jsat.classifiers.linear.LinearSGD

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, Classifier, UpdateableClassifier, Parameterized, Regressor, UpdateableRegressor, SimpleWeightVectorModel

public class LinearSGD
extends BaseUpdateableClassifier
implements UpdateableRegressor, Parameterized, SimpleWeightVectorModel

LinearSGD learns either a classification or regression problem depending on the loss function ℓ(w,x) used. The solution attempts to minimize ∑_i ℓ(w,x_i) + λ0/2 ||w||₂² + λ1 ||w||₁, and is trained by Stochastic Gradient Descent.


NOTE: To support L₁ regularization with sparse results and online learning at the same time, the normalization of the regularization penalty by the number of data points is not present in the implementation at this time. Setting λ1 to the desired value divided by the number of unique data points in the whole set will result in the correct regularization penalty being applied. See:

• Tsuruoka, Y., Tsujii, J.,&Ananiadou, S. (2009). Stochastic gradient descent training for L1-regularized log-linear models with cumulative penalty. Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 1, 477. doi:10.3115/1687878.1687946

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
LinearSGD() Creates a new LinearSGD learner for multi-class classification problems.
LinearSGD(LinearSGD toClone) Copy constructor
LinearSGD(LossFunc loss, double eta, DecayRate decay, double lambda0, double lambda1) Creates a new LinearSGD learner.
LinearSGD(LossFunc loss, double lambda0, double lambda1) Creates a new LinearSGD learner

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
LinearSGD	clone()
double	getBias(int index) Returns the bias term used with the weight vector for the given class index.
double	getEta() Returns the current learning rate in use
DecayRate	getEtaDecay() Returns the decay rate in use
GradientUpdater	getGradientUpdater()
double	getLambda0() Returns the L2 regularization term in use
double	getLambda1() Returns the L1 regularization term in use
LossFunc	getLoss() Returns the loss function in use
Vec	getRawWeight(int index) Returns the raw weight vector associated with the given class index.
static Distribution	guessLambda0(DataSet d) Guess the distribution to use for the regularization term λ0 .
static Distribution	guessLambda1(DataSet d) Guess the distribution to use for the regularization term λ1 .
boolean	isUseBias() Returns whether or not an implicit bias term is in use
int	numWeightsVecs() Returns the number of weight vectors that can be returned.
double	regress(DataPoint data)
void	setEta(double eta) Sets the initial learning rate η to use.
void	setEtaDecay(DecayRate decay) Sets the rate at which η is decayed at each update.
void	setGradientUpdater(GradientUpdater gradientUpdater) Sets the method that will be used to update the weight vectors given their gradient information.
void	setLambda0(double lambda0) λ0 controls the L2 regularization penalty.
void	setLambda1(double lambda1) λ1 controls the L1 regularization penalty.
void	setLoss(LossFunc loss) Sets the loss function used for the model.
void	setUp(CategoricalData[] categoricalAttributes, int numericAttributes) Prepares the classifier to begin learning from its UpdateableRegressor.update(jsat.classifiers.DataPoint, double) method.
void	setUp(CategoricalData[] categoricalAttributes, int numericAttributes, CategoricalData predicting) Prepares the classifier to begin learning from its UpdateableClassifier.update(jsat.classifiers.DataPoint, int) method.
void	setUseBias(boolean useBias) Sets whether or not an implicit bias term will be added to the data set
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	train(RegressionDataSet dataSet)
void	train(RegressionDataSet dataSet, boolean parallel)
void	update(DataPoint dataPoint, double targetValue) Updates the classifier by giving it a new data point to learn from.
void	update(DataPoint dataPoint, int targetClass) Updates the classifier by giving it a new data point to learn from.

Methods inherited from class jsat.classifiers.BaseUpdateableClassifier

getEpochs, setEpochs, train, train, trainEpochs

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface jsat.parameters.Parameterized

getParameter, getParameters

Constructor Detail

LinearSGD

public LinearSGD()

Creates a new LinearSGD learner for multi-class classification problems.

LinearSGD

public LinearSGD(LossFunc loss,
                 double lambda0,
                 double lambda1)

Creates a new LinearSGD learner

Parameters:

loss - the loss function to use

lambda0 - the L₂ regularization term

lambda1 - the L₁ regularization term

LinearSGD

public LinearSGD(LossFunc loss,
                 double eta,
                 DecayRate decay,
                 double lambda0,
                 double lambda1)

Creates a new LinearSGD learner.

Parameters:

loss - the loss function to use

eta - the initial learning rate

decay - the decay rate for η

lambda0 - the L₂ regularization term

lambda1 - the L₁ regularization term

LinearSGD

public LinearSGD(LinearSGD toClone)

Copy constructor

Parameters:

toClone - the object to copy

Method Detail

setGradientUpdater

public void setGradientUpdater(GradientUpdater gradientUpdater)

Sets the method that will be used to update the weight vectors given their gradient information.

Parameters:

gradientUpdater - the method to use for updating the weight vectors from the gradient

getGradientUpdater

public GradientUpdater getGradientUpdater()

Returns:

the method to use for updating the weight vectors from the gradient

setEtaDecay

public void setEtaDecay(DecayRate decay)

Sets the rate at which η is decayed at each update.

Parameters:

decay - the decay rate to use

getEtaDecay

public DecayRate getEtaDecay()

Returns the decay rate in use

Returns:

the decay rate in use

setEta

public void setEta(double eta)

Sets the initial learning rate η to use. It should generally be in (0, 1), but any positive value is acceptable.

Parameters:

eta - the learning rate to use.

getEta

public double getEta()

Returns the current learning rate in use

Returns:

the current learning rate in use

setLoss

public void setLoss(LossFunc loss)

Sets the loss function used for the model. The loss function controls whether or not regression, binary classification, or multi-class classification is supported.

Parameters:

loss - the loss function to use

getLoss

public LossFunc getLoss()

Returns the loss function in use

Returns:

the loss function in use

setLambda0

public void setLambda0(double lambda0)

λ₀ controls the L₂ regularization penalty.

Parameters:

lambda0 - the L₂ regularization penalty to use

getLambda0

public double getLambda0()

Returns the L₂ regularization term in use

Returns:

the L₂ regularization term in use

setLambda1

public void setLambda1(double lambda1)

λ₁ controls the L₁ regularization penalty.

Parameters:

lambda1 - the L₁ regularization penalty to use

getLambda1

public double getLambda1()

Returns the L₁ regularization term in use

Returns:

the L₁ regularization term in use

setUseBias

public void setUseBias(boolean useBias)

Sets whether or not an implicit bias term will be added to the data set

Parameters:

useBias - true to add an implicit bias term

isUseBias

public boolean isUseBias()

Returns whether or not an implicit bias term is in use

Returns:

true if a bias term is in use

clone

public LinearSGD clone()

Specified by:

clone in interface Classifier

Specified by:

clone in interface UpdateableClassifier

Specified by:

clone in interface Regressor

Specified by:

clone in interface UpdateableRegressor

Specified by:

clone in class BaseUpdateableClassifier

setUp

public void setUp(CategoricalData[] categoricalAttributes,
                  int numericAttributes,
                  CategoricalData predicting)

Description copied from interface: UpdateableClassifier

Prepares the classifier to begin learning from its UpdateableClassifier.update(jsat.classifiers.DataPoint, int) method.

Specified by:

setUp in interface UpdateableClassifier

Parameters:

categoricalAttributes - an array containing the categorical attributes that will be in each data point

numericAttributes - the number of numeric attributes that will be in each data point

predicting - the information for the target class that will be predicted

setUp

public void setUp(CategoricalData[] categoricalAttributes,
                  int numericAttributes)

Description copied from interface: UpdateableRegressor

Prepares the classifier to begin learning from its UpdateableRegressor.update(jsat.classifiers.DataPoint, double) method.

Specified by:

setUp in interface UpdateableRegressor

Parameters:

categoricalAttributes - an array containing the categorical attributes that will be in each data point

numericAttributes - the number of numeric attributes that will be in each data point

update

public void update(DataPoint dataPoint,
                   int targetClass)

Description copied from interface: UpdateableClassifier

Updates the classifier by giving it a new data point to learn from.

Specified by:

update in interface UpdateableClassifier

Parameters:

dataPoint - the data point to learn

targetClass - the target class of the data point

update

public void update(DataPoint dataPoint,
                   double targetValue)

Description copied from interface: UpdateableRegressor

Updates the classifier by giving it a new data point to learn from.

Specified by:

update in interface UpdateableRegressor

Parameters:

dataPoint - the data point to learn

targetValue - the target value of the data point

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Specified by:

classify in interface Classifier

Parameters:

data - the data point to classify

Returns:

the results of the classification.

regress

public double regress(DataPoint data)

Specified by:

regress in interface Regressor

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Specified by:

supportsWeightedData in interface Classifier

Specified by:

supportsWeightedData in interface Regressor

Returns:

true if the model supports weighted data, false otherwise

train

public void train(RegressionDataSet dataSet,
                  boolean parallel)

Specified by:

train in interface Regressor

train

public void train(RegressionDataSet dataSet)

Specified by:

train in interface Regressor

getRawWeight

public Vec getRawWeight(int index)

Description copied from interface: SimpleWeightVectorModel

Returns the raw weight vector associated with the given class index. If the given class is an implicit zero vector, a ConstantVector object may be returned.
Do not alter the returned weight vector, as it will change the model's values.

If a regression problem, only index = 0 should be used

Specified by:

getRawWeight in interface SimpleWeightVectorModel

Parameters:

index - the class index to get the weight vector for

Returns:

the weight vector used for the specified class

getBias

public double getBias(int index)

Description copied from interface: SimpleWeightVectorModel

Returns the bias term used with the weight vector for the given class index. If the model does not support or was not trained with bias weights, 0 will be returned.

If a regression problem, only index = 0 should be used

Specified by:

getBias in interface SimpleWeightVectorModel

Parameters:

index - the class index to get the weight vector for

Returns:

the bias term for the specified class

numWeightsVecs

public int numWeightsVecs()

Description copied from interface: SimpleWeightVectorModel

Returns the number of weight vectors that can be returned. For binary classification problems the value may be 1 if only a single weight vector's sign is used to determine the class. For multi-class problems, the weight vector count includes the implicit zero vector (if one is being used).

Specified by:

numWeightsVecs in interface SimpleWeightVectorModel

Returns:

the number of weight vectors for which SimpleWeightVectorModel.getRawWeight(int) can be called.

guessLambda0

public static Distribution guessLambda0(DataSet d)

Guess the distribution to use for the regularization term λ0 .

Parameters:

d - the data set to get the guess for

Returns:

the guess for the λ₀ parameter

guessLambda1

public static Distribution guessLambda1(DataSet d)

Guess the distribution to use for the regularization term λ1 .

Parameters:

d - the data set to get the guess for

Returns:

the guess for the λ₁ parameter