jsat.classifiers.linear

Class SMIDAS

java.lang.Object

jsat.classifiers.linear.StochasticSTLinearL1

jsat.classifiers.linear.SMIDAS

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, Classifier, Parameterized, Regressor, SimpleWeightVectorModel, SingleWeightVectorModel

public class SMIDAS
extends StochasticSTLinearL1

Implements the iterative and single threaded stochastic solver for L₁ regularized linear regression problems SMIDAS (Stochastic Mirror Descent Algorithm mAde Sparse). It performs very well when the number of features is large relative to or greater than the number of data points. It also works decently on smaller sparse data sets.
Using the squared loss is equivalent to LASSO regression, and the LOG loss is equivalent to logistic regression.

Note: This implementation requires all feature values to be in the range [-1, 1]. By default scaling is performed to [0,1] to preserve sparseness. If your data is dense or has negative and positive feature values, scaling to [-1, 1] may perform better. See StochasticSTLinearL1.setReScale(boolean)

See:
Shalev-Shwartz, S.,&Tewari, A. (2009). Stochastic Methods for L₁-regularized Loss Minimization. 26th International Conference on Machine Learning (Vol. 12, pp. 929–936).

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class jsat.classifiers.linear.StochasticSTLinearL1

StochasticSTLinearL1.Loss

Field Summary

Fields inherited from class jsat.classifiers.linear.StochasticSTLinearL1

DEFAULT_EPOCHS, DEFAULT_LOSS, DEFAULT_REG

Constructor Summary

Constructors

Constructor and Description
SMIDAS(double eta) Creates a new SMIDAS learner
SMIDAS(double eta, int epochs, double lambda, StochasticSTLinearL1.Loss loss) Creates a new SMIDAS learner
SMIDAS(double eta, int epochs, double lambda, StochasticSTLinearL1.Loss loss, boolean reScale) Creates a new SMIDAS learner

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
SMIDAS	clone()
double	getEta() Returns the current learning rate used during training
double	regress(DataPoint data)
void	setEta(double eta) Sets the learning rate used during training
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	train(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	train(ClassificationDataSet dataSet, boolean parallel) Trains the classifier and constructs a model for classification using the given data set.
void	train(RegressionDataSet dataSet)
void	train(RegressionDataSet dataSet, boolean parallel)

Methods inherited from class jsat.classifiers.linear.StochasticSTLinearL1

getBias, getBias, getEpochs, getLambda, getLoss, getMaxScaled, getMinScaled, getRawWeight, getRawWeight, getW, getWRaw, isReScale, numWeightsVecs, setEpochs, setLambda, setLoss, setMaxScaled, setMinScaled, setReScale

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface jsat.parameters.Parameterized

getParameter, getParameters

Constructor Detail

SMIDAS

public SMIDAS(double eta)

Creates a new SMIDAS learner

Parameters:

eta - the learning rate for each iteration

SMIDAS

public SMIDAS(double eta,
              int epochs,
              double lambda,
              StochasticSTLinearL1.Loss loss)

Creates a new SMIDAS learner

Parameters:

eta - the learning rate for each iteration

epochs - the number of learning iterations

lambda - the regularization penalty

loss - the loss function to use

SMIDAS

public SMIDAS(double eta,
              int epochs,
              double lambda,
              StochasticSTLinearL1.Loss loss,
              boolean reScale)

Creates a new SMIDAS learner

Parameters:

eta - the learning rate for each iteration

epochs - the number of learning iterations

lambda - the regularization penalty

loss - the loss function to use

reScale - whether or not to rescale the feature values

Method Detail

setEta

public void setEta(double eta)

Sets the learning rate used during training

Parameters:

eta - the learning rate to use

getEta

public double getEta()

Returns the current learning rate used during training

Returns:

the learning rate in use

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Parameters:

data - the data point to classify

Returns:

the results of the classification.

regress

public double regress(DataPoint data)

train

public void train(ClassificationDataSet dataSet,
                  boolean parallel)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Parameters:

dataSet - the data set to train on

parallel - true if multiple threads should be used to train the model. false if it should be done in a single threaded manner.

train

public void train(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Parameters:

dataSet - the data set to train on

train

public void train(RegressionDataSet dataSet,
                  boolean parallel)

train

public void train(RegressionDataSet dataSet)

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Returns:

true if the model supports weighted data, false otherwise

clone

public SMIDAS clone()

Specified by:

clone in interface Classifier

Specified by:

clone in interface Regressor

Specified by:

clone in class StochasticSTLinearL1