jsat.classifiers.linear

Class StochasticSTLinearL1

java.lang.Object

jsat.classifiers.linear.StochasticSTLinearL1

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, Classifier, Parameterized, Regressor, SimpleWeightVectorModel, SingleWeightVectorModel

Direct Known Subclasses:

LinearL1SCD, SMIDAS

public abstract class StochasticSTLinearL1
extends java.lang.Object
implements Classifier, Regressor, Parameterized, SingleWeightVectorModel

This base class provides shared functionality and variables used by two different training algorithms for L₁ regularized linear models. Both squared and log loss are supported, making the results equivalent to LASSO regression and Logistic regression respectively.

These algorithms requires all feature values to be in the range [-1, 1]. The implementation can do implicit rescaling, but rescaling may destroy sparsity. If the data set is sparse and all values are zero or positive use the default [0,1] rescaling to perform efficient rescaling that will not destroy sparsity.

Both algorithms are from:
Shalev-Shwartz, S.,&Tewari, A. (2009). Stochastic Methods for L₁-regularized Loss Minimization. 26th International Conference on Machine Learning (Vol. 12, pp. 929–936).

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	StochasticSTLinearL1.Loss

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_EPOCHS
static StochasticSTLinearL1.Loss	DEFAULT_LOSS
static double	DEFAULT_REG

Constructor Summary

Constructors

Constructor and Description
StochasticSTLinearL1()

Method Summary

Modifier and Type	Method and Description
abstract StochasticSTLinearL1	clone()
double	getBias() Returns the bias term used for the model, or 0 of the model does not support or was not trained with a bias term.
double	getBias(int index) Returns the bias term used with the weight vector for the given class index.
double	getEpochs() Returns the number of iterations of updating that will be done
double	getLambda() Returns the amount of regularization to used in training
StochasticSTLinearL1.Loss	getLoss() returns the loss function in use
double	getMaxScaled() Returns the maximum feature value after scaling
double	getMinScaled() Returns the minimum feature value after scaling
Vec	getRawWeight() Returns the only weight vector used for the model
Vec	getRawWeight(int index) Returns the raw weight vector associated with the given class index.
Vec	getW() Returns a copy of the weight vector used to compute results via a dot product.
Vec	getWRaw() Returns the weight vector used to compute results via a dot product.
boolean	isReScale() Returns if scaling is in use
int	numWeightsVecs() Returns the number of weight vectors that can be returned.
void	setEpochs(int epochs) Sets the number of iterations of training that will be performed.
void	setLambda(double lambda) Sets the regularization constant used for learning.
void	setLoss(StochasticSTLinearL1.Loss loss) Sets the loss function to use.
void	setMaxScaled(double maxFeature) Sets the maximum value of any feature after scaling is applied.
void	setMinScaled(double minFeature) Sets the minimum value of any feature after scaling is applied.
void	setReScale(boolean reScale) Sets whether or not scaling should be applied on th feature values of the training vectors.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface jsat.classifiers.Classifier

classify, supportsWeightedData, train, train

Methods inherited from interface jsat.regression.Regressor

regress, supportsWeightedData, train, train

Methods inherited from interface jsat.parameters.Parameterized

getParameter, getParameters

Field Detail

DEFAULT_EPOCHS

public static final int DEFAULT_EPOCHS

See Also:

Constant Field Values

DEFAULT_REG

public static final double DEFAULT_REG

See Also:

Constant Field Values

DEFAULT_LOSS

public static final StochasticSTLinearL1.Loss DEFAULT_LOSS

Constructor Detail

StochasticSTLinearL1

public StochasticSTLinearL1()

Method Detail

clone

public abstract StochasticSTLinearL1 clone()

Specified by:

clone in interface Classifier

Specified by:

clone in interface Regressor

Overrides:

clone in class java.lang.Object

setEpochs

public void setEpochs(int epochs)

Sets the number of iterations of training that will be performed.

Parameters:

epochs - the number of iterations

getEpochs

public double getEpochs()

Returns the number of iterations of updating that will be done

Returns:

the number of iterations

setMaxScaled

public void setMaxScaled(double maxFeature)

Sets the maximum value of any feature after scaling is applied. This value can be no greater than 1.

Parameters:

maxFeature - the maximum feature value after scaling

getMaxScaled

public double getMaxScaled()

Returns the maximum feature value after scaling

Returns:

the maximum feature value after scaling

setMinScaled

public void setMinScaled(double minFeature)

Sets the minimum value of any feature after scaling is applied. This value can be no smaller than -1

Parameters:

minFeature - the minimum feature value after scaling

getMinScaled

public double getMinScaled()

Returns the minimum feature value after scaling

Returns:

the minimum feature value after scaling

setLambda

public void setLambda(double lambda)

Sets the regularization constant used for learning. The regularization must be positive, and the learning rate is proportional to the regularization value. This means regularizations very near zero will take a long time to converge.

Parameters:

lambda - the regularization to apply

getLambda

public double getLambda()

Returns the amount of regularization to used in training

Returns:

the regularization parameter.

setLoss

public void setLoss(StochasticSTLinearL1.Loss loss)

Sets the loss function to use. This should not be altered after training unless the leaner is going to be trained again.

Parameters:

loss - the loss function to use

getLoss

public StochasticSTLinearL1.Loss getLoss()

returns the loss function in use

Returns:

the loss function in use

setReScale

public void setReScale(boolean reScale)

Sets whether or not scaling should be applied on th feature values of the training vectors. Scaling should be used intelligently, scaling can destroy sparsity in the data set. If scaling is not applied, and a value is not in the range [-1, 1], a FailedToFitException could occur.
Rescaling does not alter the data points passed in.

Parameters:

reScale - whether or not to rescale feature values

isReScale

public boolean isReScale()

Returns if scaling is in use

Returns:

true if feature values are rescaled during training.

getWRaw

public Vec getWRaw()

Returns the weight vector used to compute results via a dot product.
Do not modify this value, or you will alter the results returned.

Returns:

the learned weight vector for prediction

getW

public Vec getW()

Returns a copy of the weight vector used to compute results via a dot product.

Returns:

a copy of the learned weight vector for prediction

getRawWeight

public Vec getRawWeight()

Description copied from interface: SingleWeightVectorModel

Returns the only weight vector used for the model

Specified by:

getRawWeight in interface SingleWeightVectorModel

Returns:

the only weight vector used for the model

getBias

public double getBias()

Description copied from interface: SingleWeightVectorModel

Returns the bias term used for the model, or 0 of the model does not support or was not trained with a bias term.

Specified by:

getBias in interface SingleWeightVectorModel

Returns:

the bias term for the model

getRawWeight

public Vec getRawWeight(int index)

Description copied from interface: SimpleWeightVectorModel

Returns the raw weight vector associated with the given class index. If the given class is an implicit zero vector, a ConstantVector object may be returned.
Do not alter the returned weight vector, as it will change the model's values.

If a regression problem, only index = 0 should be used

Specified by:

getRawWeight in interface SimpleWeightVectorModel

Parameters:

index - the class index to get the weight vector for

Returns:

the weight vector used for the specified class

getBias

public double getBias(int index)

Description copied from interface: SimpleWeightVectorModel

Returns the bias term used with the weight vector for the given class index. If the model does not support or was not trained with bias weights, 0 will be returned.

If a regression problem, only index = 0 should be used

Specified by:

getBias in interface SimpleWeightVectorModel

Parameters:

index - the class index to get the weight vector for

Returns:

the bias term for the specified class

numWeightsVecs

public int numWeightsVecs()

Description copied from interface: SimpleWeightVectorModel

Returns the number of weight vectors that can be returned. For binary classification problems the value may be 1 if only a single weight vector's sign is used to determine the class. For multi-class problems, the weight vector count includes the implicit zero vector (if one is being used).

Specified by:

numWeightsVecs in interface SimpleWeightVectorModel

Returns:

the number of weight vectors for which SimpleWeightVectorModel.getRawWeight(int) can be called.