jsat.classifiers.linear

Class SCW

java.lang.Object

jsat.classifiers.BaseUpdateableClassifier

jsat.classifiers.linear.SCW

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, BinaryScoreClassifier, Classifier, UpdateableClassifier, Parameterized, SimpleWeightVectorModel, SingleWeightVectorModel

public class SCW
extends BaseUpdateableClassifier
implements BinaryScoreClassifier, Parameterized, SingleWeightVectorModel

Provides an Implementation of Confidence-Weighted (CW) learning and Soft Confidence-Weighted (SCW), both of which are binary linear classifiers inspired by PassiveAggressive. The SCW mode handles noisy and nonlinearly separable datasets better.
NOTE: Unlike other online second order methods, when using the full covariance matrix, all new inputs cost O(d²) time to process, even if update is needed.
NOTE: This implementation does not add an implicit bias term, so the solution goes through the origin

See:

• Crammer, K., Fern, M.,&Pereira, O. (2008). Exact Convex Confidence-Weighted Learning. In Advances in Neural Information Processing Systems 22 (pp. 345–352). Retrieved from here
• Wang, J., Zhao, P.,&Hoi, S. C. H. (2012). Exact Soft Confidence-Weighted Learning. ICML. Learning. Retrieved from here

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SCW.Mode Which version of the algorithms shuld be used

Constructor Summary

Constructors

Constructor and Description
SCW() Creates a new SCW learner
SCW(double eta, SCW.Mode mode, boolean diagonalOnly) Creates a new SCW learner

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
SCW	clone()
double	getBias() Returns the bias term used for the model, or 0 of the model does not support or was not trained with a bias term.
double	getBias(int index) Returns the bias term used with the weight vector for the given class index.
double	getC() Returns the aggressiveness parameter
double	getEta() Returns the target correction confidence
SCW.Mode	getMode() Returns which algorithm is used
Vec	getRawWeight() Returns the only weight vector used for the model
Vec	getRawWeight(int index) Returns the raw weight vector associated with the given class index.
double	getScore(DataPoint dp) Returns the numeric score for predicting a class of a given data point, where the sign of the value indicates which class the data point is predicted to belong to.
Vec	getWeightVec() Returns the weight vector used to compute results via a dot product.
static Distribution	guessC(DataSet d) Guess the distribution to use for the regularization term C .
static Distribution	guessEta(DataSet d) Guess the distribution to use for the regularization term η .
boolean	isDiagonalOnly() Returns true if the covariance matrix is restricted to its diagonal entries
int	numWeightsVecs() Returns the number of weight vectors that can be returned.
void	setC(double C) Set the aggressiveness parameter.
void	setDiagonalOnly(boolean diagonalOnly) Using the full covariance matrix requires O(d2) work on updates, where d is the dimension of the data.
void	setEta(double eta) SCW uses a probabilistic version of the margin and attempts to make a correction so that the confidence with correct label would be of a certain threshold, which is set by eta.
void	setMode(SCW.Mode mode) Controls which version of the algorithm is used
void	setUp(CategoricalData[] categoricalAttributes, int numericAttributes, CategoricalData predicting) Prepares the classifier to begin learning from its UpdateableClassifier.update(jsat.classifiers.DataPoint, int) method.
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	update(DataPoint dataPoint, int targetClass) Updates the classifier by giving it a new data point to learn from.

Methods inherited from class jsat.classifiers.BaseUpdateableClassifier

getEpochs, setEpochs, train, train, trainEpochs

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface jsat.classifiers.Classifier

train, train

Methods inherited from interface jsat.parameters.Parameterized

getParameter, getParameters

Constructor Detail

SCW

public SCW()

Creates a new SCW learner

SCW

public SCW(double eta,
           SCW.Mode mode,
           boolean diagonalOnly)

Creates a new SCW learner

Parameters:

eta - the margin confidence parameter in [0.5, 1]

mode - mode controlling which algorithm to use

diagonalOnly - whether or not to use only the diagonal of the covariance matrix

See Also:

setEta(double), setMode(jsat.classifiers.linear.SCW.Mode), setDiagonalOnly(boolean)

Method Detail

setEta

public void setEta(double eta)

SCW uses a probabilistic version of the margin and attempts to make a correction so that the confidence with correct label would be of a certain threshold, which is set by eta. So the threshold must be in [0.5, 1.0]. Values in the range [0.8, 0.9] often work well on a wide range of problems

Parameters:

eta - the confidence to correct to

getEta

public double getEta()

Returns the target correction confidence

Returns:

the target correction confidence

setC

public void setC(double C)

Set the aggressiveness parameter. Increasing the value of this parameter increases the aggressiveness of the algorithm. It must be a positive value. This parameter essentially performs a type of regularization on the updates
The aggressiveness parameter is only used by SCW.Mode.SCWI and SCW.Mode.SCWII

Parameters:

C - the positive aggressiveness parameter

getC

public double getC()

Returns the aggressiveness parameter

Returns:

the aggressiveness parameter

setMode

public void setMode(SCW.Mode mode)

Controls which version of the algorithm is used

Parameters:

mode - which algorithm to use

getMode

public SCW.Mode getMode()

Returns which algorithm is used

Returns:

which algorithm is used

setDiagonalOnly

public void setDiagonalOnly(boolean diagonalOnly)

Using the full covariance matrix requires O(d²) work on updates, where d is the dimension of the data. Runtime can be reduced by using only the diagonal of the matrix to perform updates in O(s) time, where s ≤ d is the number of non-zero values in the input

Parameters:

diagonalOnly - true to use only the diagonal of the covariance

isDiagonalOnly

public boolean isDiagonalOnly()

Returns true if the covariance matrix is restricted to its diagonal entries

Returns:

true if the covariance matrix is restricted to its diagonal entries

getWeightVec

public Vec getWeightVec()

Returns the weight vector used to compute results via a dot product.
Do not modify this value, or you will alter the results returned.

Returns:

the learned weight vector for prediction

getRawWeight

public Vec getRawWeight()

Description copied from interface: SingleWeightVectorModel

Returns the only weight vector used for the model

Specified by:

getRawWeight in interface SingleWeightVectorModel

Returns:

the only weight vector used for the model

getBias

public double getBias()

Description copied from interface: SingleWeightVectorModel

Returns the bias term used for the model, or 0 of the model does not support or was not trained with a bias term.

Specified by:

getBias in interface SingleWeightVectorModel

Returns:

the bias term for the model

getRawWeight

public Vec getRawWeight(int index)

Description copied from interface: SimpleWeightVectorModel

Returns the raw weight vector associated with the given class index. If the given class is an implicit zero vector, a ConstantVector object may be returned.
Do not alter the returned weight vector, as it will change the model's values.

If a regression problem, only index = 0 should be used

Specified by:

getRawWeight in interface SimpleWeightVectorModel

Parameters:

index - the class index to get the weight vector for

Returns:

the weight vector used for the specified class

getBias

public double getBias(int index)

Description copied from interface: SimpleWeightVectorModel

Returns the bias term used with the weight vector for the given class index. If the model does not support or was not trained with bias weights, 0 will be returned.

If a regression problem, only index = 0 should be used

Specified by:

getBias in interface SimpleWeightVectorModel

Parameters:

index - the class index to get the weight vector for

Returns:

the bias term for the specified class

numWeightsVecs

public int numWeightsVecs()

Description copied from interface: SimpleWeightVectorModel

Returns the number of weight vectors that can be returned. For binary classification problems the value may be 1 if only a single weight vector's sign is used to determine the class. For multi-class problems, the weight vector count includes the implicit zero vector (if one is being used).

Specified by:

numWeightsVecs in interface SimpleWeightVectorModel

Returns:

the number of weight vectors for which SimpleWeightVectorModel.getRawWeight(int) can be called.

clone

public SCW clone()

Specified by:

clone in interface BinaryScoreClassifier

Specified by:

clone in interface Classifier

Specified by:

clone in interface UpdateableClassifier

Specified by:

clone in class BaseUpdateableClassifier

setUp

public void setUp(CategoricalData[] categoricalAttributes,
                  int numericAttributes,
                  CategoricalData predicting)

Description copied from interface: UpdateableClassifier

Prepares the classifier to begin learning from its UpdateableClassifier.update(jsat.classifiers.DataPoint, int) method.

Specified by:

setUp in interface UpdateableClassifier

Parameters:

categoricalAttributes - an array containing the categorical attributes that will be in each data point

numericAttributes - the number of numeric attributes that will be in each data point

predicting - the information for the target class that will be predicted

update

public void update(DataPoint dataPoint,
                   int targetClass)

Description copied from interface: UpdateableClassifier

Updates the classifier by giving it a new data point to learn from.

Specified by:

update in interface UpdateableClassifier

Parameters:

dataPoint - the data point to learn

targetClass - the target class of the data point

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Specified by:

classify in interface Classifier

Parameters:

data - the data point to classify

Returns:

the results of the classification.

getScore

public double getScore(DataPoint dp)

Description copied from interface: BinaryScoreClassifier

Returns the numeric score for predicting a class of a given data point, where the sign of the value indicates which class the data point is predicted to belong to.

Specified by:

getScore in interface BinaryScoreClassifier

Parameters:

dp - the data point to predict the class label of

Returns:

the score for the given data point

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Specified by:

supportsWeightedData in interface Classifier

Returns:

true if the model supports weighted data, false otherwise

guessC

public static Distribution guessC(DataSet d)

Guess the distribution to use for the regularization term C .

Parameters:

d - the data set to get the guess for

Returns:

the guess for the C parameter

guessEta

public static Distribution guessEta(DataSet d)

Guess the distribution to use for the regularization term η .

Parameters:

d - the data set to get the guess for

Returns:

the guess for the C parameter