jsat.classifiers.linear.kernelized

Class CSKLRBatch

java.lang.Object

jsat.classifiers.svm.SupportVectorLearner

jsat.classifiers.linear.kernelized.CSKLRBatch

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, Classifier, Parameterized

public class CSKLRBatch
extends SupportVectorLearner
implements Parameterized, Classifier

An implementation of Conservative Stochastic Kernel Logistic Regression. This is an online algorithm that obtains sparse solutions by conservatively rejecting updates based on a binomial distribution of the error on each update.

This algorithm works best on data sets with a very high number of samples where a high accuracy is obtainable using a kernel model. It is often the case that this model produces accurate results, but has a low confidence due to the conservative updating. This can be counteracted by having a very large number of features, but that often increases the size of the model.
This batch version can also be used to more efficiently learn dense KLR models using the stochastic method with the CSKLR.UpdateMode.NC mode if model sparsity is not important.

It is important to read the documentation and test some different values for the learning rate and gamma variables. They behave different compared to many algorithms.

See paper:
Zhang, L., Jin, R., Chen, C., Bu, J.,&He, X. (2012). Efficient Online Learning for Large-Scale Sparse Kernel Logistic Regression. Twenty-Sixth AAAI Conference on Artificial Intelligence (pp. 1219–1225). Retrieved from here

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class jsat.classifiers.svm.SupportVectorLearner

SupportVectorLearner.CacheMode

Constructor Summary

Constructors

Constructor and Description
CSKLRBatch(double eta, KernelTrick kernel, double R, CSKLR.UpdateMode mode, SupportVectorLearner.CacheMode cacheMode) Creates a new SCKLR Batch learning object

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
CSKLRBatch	clone()
int	getEpochs() Returns the number of passes through the data set
double	getEta() Returns the learning rate to use
double	getGamma() Returns the gamma sparsity parameter value
CSKLR.UpdateMode	getMode() Returns the update mode in use
double	getR() Returns the maximal norm of the algorithm
static Distribution	guessR(DataSet d) Guesses the distribution to use for the R parameter
void	setEpochs(int epochs) Sets the number of training epochs (passes) through the data set
void	setEta(double eta) Sets the learning rate to use for the algorithm.
void	setGamma(double gamma) Sets the gamma value to use.
void	setMode(CSKLR.UpdateMode mode) Sets what update mode should be used.
void	setR(double R) Sets the maximal margin norm value for the algorithm.
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	train(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	train(ClassificationDataSet dataSet, boolean parallel) Trains the classifier and constructs a model for classification using the given data set.

Methods inherited from class jsat.classifiers.svm.SupportVectorLearner

getCacheMode, getCacheValue, getKernel, setCacheMode, setCacheSize, setCacheValue, setKernel

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface jsat.parameters.Parameterized

getParameter, getParameters

Constructor Detail

CSKLRBatch

public CSKLRBatch(double eta,
                  KernelTrick kernel,
                  double R,
                  CSKLR.UpdateMode mode,
                  SupportVectorLearner.CacheMode cacheMode)

Creates a new SCKLR Batch learning object

Parameters:

eta - the learning rate to use

kernel - the kernel to use

R - the maximal norm of the surface

mode - the mode to use

cacheMode - the kernel caching mode to use

Method Detail

clone

public CSKLRBatch clone()

Specified by:

clone in interface Classifier

Overrides:

clone in class java.lang.Object

setEpochs

public void setEpochs(int epochs)

Sets the number of training epochs (passes) through the data set

Parameters:

epochs - the number of passes through the data set

getEpochs

public int getEpochs()

Returns the number of passes through the data set

Returns:

the number of passes through the data set

setEta

public void setEta(double eta)

Sets the learning rate to use for the algorithm. Unlike many other stochastic algorithms, the learning rate for CSKLR should be large, often in the range of (0.5, 1) - and can even be larger than 1 at times. If the learning rate is too low, it may be difficult to get strong confidence results from the algorithm.

Parameters:

eta - the positive learning rate to use

getEta

public double getEta()

Returns the learning rate to use

Returns:

the learning rate to use

setR

public void setR(double R)

Sets the maximal margin norm value for the algorithm. When the norm is exceeded, the coefficients will be rescaled to fit in the norm. If the maximal norm is too small (less than 5), it may be difficult to get strong confidence results from the algorithm.
A good range of values suggested by the original paper is 10^x ∀ x ∈ {0, 1, 2, 3, 4, 5}

Parameters:

R -

getR

public double getR()

Returns the maximal norm of the algorithm

Returns:

the maximal norm of the algorithm

setMode

public void setMode(CSKLR.UpdateMode mode)

Sets what update mode should be used. The update mode controls the sparsity of the mode, and the behavior of setGamma(double)

Parameters:

mode - the update mode to use

getMode

public CSKLR.UpdateMode getMode()

Returns the update mode in use

Returns:

the update mode in use

setGamma

public void setGamma(double gamma)

Sets the gamma value to use. This value, depending on which CSKLR.UpdateMode is used, controls the sparsity of the model.

Parameters:

gamma - the gamma parameter, which is at least always positive

getGamma

public double getGamma()

Returns the gamma sparsity parameter value

Returns:

the gamma sparsity parameter value

guessR

public static Distribution guessR(DataSet d)

Guesses the distribution to use for the R parameter

Parameters:

d - the dataset to get the guess for

Returns:

the guess for the R parameter

See Also:

setR(double)

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Specified by:

classify in interface Classifier

Parameters:

data - the data point to classify

Returns:

the results of the classification.

train

public void train(ClassificationDataSet dataSet,
                  boolean parallel)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Specified by:

train in interface Classifier

Parameters:

dataSet - the data set to train on

parallel - true if multiple threads should be used to train the model. false if it should be done in a single threaded manner.

train

public void train(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Specified by:

train in interface Classifier

Parameters:

dataSet - the data set to train on

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Specified by:

supportsWeightedData in interface Classifier

Returns:

true if the model supports weighted data, false otherwise