jsat.classifiers.linear.kernelized

Class KernelSGD

java.lang.Object

jsat.classifiers.linear.kernelized.KernelSGD

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, Classifier, UpdateableClassifier, Parameterized, Regressor, UpdateableRegressor

public class KernelSGD
extends java.lang.Object
implements UpdateableClassifier, UpdateableRegressor, Parameterized

Kernel SGD is the kernelized counterpart to LinearSGD, and learns nonlinear functions via the kernel trick. The implementation is built upon KernelPoint and KernelPoints to support budgeted learning. Following the LinearSGD implementation, whether or not this algorithm supports regression, binary-classification, or multi-class classification is controlled by the loss function used.

The learning rate decay is not configurable for this implementation, and is decayed at a rate of η / (λ * (t + 2 / λ)) , where t is the time step.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
KernelSGD() Creates a new Kernel SGD object for classification with the RBF kernel
KernelSGD(KernelSGD toCopy) Copy constructor
KernelSGD(LossFunc loss, KernelTrick kernel, double lambda, KernelPoint.BudgetStrategy budgetStrategy, int budgetSize) Creates a new Kernel SGD object
KernelSGD(LossFunc loss, KernelTrick kernel, double lambda, KernelPoint.BudgetStrategy budgetStrategy, int budgetSize, double eta, double errorTolerance) Creates a new Kernel SGD object

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
KernelSGD	clone()
int	getBudgetSize() Returns the budget size, or maximum number of allowed support vectors.
KernelPoint.BudgetStrategy	getBudgetStrategy() Returns the method of budget maintenance
int	getEpochs() Returns the number of epochs to use
double	getErrorTolerance() Returns the error tolerance that would be used
double	getEta() Returns the base learning rate
KernelTrick	getKernel() Returns the kernel in use
double	getLambda() Returns the L2 regularization parameter
LossFunc	getLoss() Returns the loss function in use
static Distribution	guessLambda(DataSet d) Guess the distribution to use for the regularization term λ .
double	regress(DataPoint data)
void	setBudgetSize(int budgetSize) Sets the maximum budget size, or number of support vectors, to allow during training.
void	setBudgetStrategy(KernelPoint.BudgetStrategy budgetStrategy) Sets the budget maintenance strategy.
void	setEpochs(int epochs) Sets the number of iterations of the training set done during batch training
void	setErrorTolerance(double errorTolerance) Sets the error tolerance used for certain budget strategies
void	setEta(double eta) Sets the base learning rate to start from.
void	setKernel(KernelTrick kernel) Sets the kernel to use
void	setLambda(double lambda) Sets the L2 regularization parameter used during learning.
void	setLoss(LossFunc loss) Sets the loss function to use.
void	setUp(CategoricalData[] categoricalAttributes, int numericAttributes) Prepares the classifier to begin learning from its UpdateableRegressor.update(jsat.classifiers.DataPoint, double) method.
void	setUp(CategoricalData[] categoricalAttributes, int numericAttributes, CategoricalData predicting) Prepares the classifier to begin learning from its UpdateableClassifier.update(jsat.classifiers.DataPoint, int) method.
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	train(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	train(ClassificationDataSet dataSet, boolean parallel) Trains the classifier and constructs a model for classification using the given data set.
void	train(RegressionDataSet dataSet)
void	train(RegressionDataSet dataSet, boolean parallel)
void	update(DataPoint dataPoint, double targetValue) Updates the classifier by giving it a new data point to learn from.
void	update(DataPoint dataPoint, int targetClass) Updates the classifier by giving it a new data point to learn from.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface jsat.parameters.Parameterized

getParameter, getParameters

Constructor Detail

KernelSGD

public KernelSGD()

Creates a new Kernel SGD object for classification with the RBF kernel

KernelSGD

public KernelSGD(LossFunc loss,
                 KernelTrick kernel,
                 double lambda,
                 KernelPoint.BudgetStrategy budgetStrategy,
                 int budgetSize)

Creates a new Kernel SGD object

Parameters:

loss - the loss function to use

kernel - the kernel trick to use

lambda - the regularization penalty

budgetStrategy - the budget maintenance strategy to use

budgetSize - the maximum support vector budget

KernelSGD

public KernelSGD(LossFunc loss,
                 KernelTrick kernel,
                 double lambda,
                 KernelPoint.BudgetStrategy budgetStrategy,
                 int budgetSize,
                 double eta,
                 double errorTolerance)

Creates a new Kernel SGD object

Parameters:

loss - the loss function to use

kernel - the kernel trick to use

lambda - the regularization penalty

eta - the initial learning rate

budgetStrategy - the budget maintenance strategy to use

errorTolerance - the error tolerance used in certain budget maintenance steps

budgetSize - the maximum support vector budget

KernelSGD

public KernelSGD(KernelSGD toCopy)

Copy constructor

Parameters:

toCopy - the object to copy

Method Detail

setEpochs

public void setEpochs(int epochs)

Sets the number of iterations of the training set done during batch training

Parameters:

epochs - the number of iterations in batch training

getEpochs

public int getEpochs()

Returns the number of epochs to use

Returns:

the number of epochs to use

setLoss

public void setLoss(LossFunc loss)

Sets the loss function to use. The loss function controls whether or not classification or regression is supported.

Parameters:

loss -

getLoss

public LossFunc getLoss()

Returns the loss function in use

Returns:

the loss function in use

setLambda

public void setLambda(double lambda)

Sets the L₂ regularization parameter used during learning.

Parameters:

lambda - the positive regularization parameter

getLambda

public double getLambda()

Returns the L₂ regularization parameter

Returns:

the L₂ regularization parameter

setErrorTolerance

public void setErrorTolerance(double errorTolerance)

Sets the error tolerance used for certain budget strategies

Parameters:

errorTolerance - the error tolerance in [0, 1]

getErrorTolerance

public double getErrorTolerance()

Returns the error tolerance that would be used

Returns:

the error tolerance that would be used

setBudgetSize

public void setBudgetSize(int budgetSize)

Sets the maximum budget size, or number of support vectors, to allow during training. Increasing the budget size will increase the accuracy of the model, but will also increase the computational cost

Parameters:

budgetSize - the maximum allowed number of support vectors

getBudgetSize

public int getBudgetSize()

Returns the budget size, or maximum number of allowed support vectors.

Returns:

the maximum number of allowed support vectors

setBudgetStrategy

public void setBudgetStrategy(KernelPoint.BudgetStrategy budgetStrategy)

Sets the budget maintenance strategy.

Parameters:

budgetStrategy - the method to meet budget size requirements

getBudgetStrategy

public KernelPoint.BudgetStrategy getBudgetStrategy()

Returns the method of budget maintenance

Returns:

the method of budget maintenance

setEta

public void setEta(double eta)

Sets the base learning rate to start from. Because of the decay rate in use a good value for η is 1.0.

Parameters:

eta - the starting learning rate to use

getEta

public double getEta()

Returns the base learning rate

Returns:

the base learning rate

setKernel

public void setKernel(KernelTrick kernel)

Sets the kernel to use

Parameters:

kernel - the kernel to use

getKernel

public KernelTrick getKernel()

Returns the kernel in use

Returns:

the kernel in use

setUp

public void setUp(CategoricalData[] categoricalAttributes,
                  int numericAttributes,
                  CategoricalData predicting)

Description copied from interface: UpdateableClassifier

Prepares the classifier to begin learning from its UpdateableClassifier.update(jsat.classifiers.DataPoint, int) method.

Specified by:

setUp in interface UpdateableClassifier

Parameters:

categoricalAttributes - an array containing the categorical attributes that will be in each data point

numericAttributes - the number of numeric attributes that will be in each data point

predicting - the information for the target class that will be predicted

setUp

public void setUp(CategoricalData[] categoricalAttributes,
                  int numericAttributes)

Description copied from interface: UpdateableRegressor

Prepares the classifier to begin learning from its UpdateableRegressor.update(jsat.classifiers.DataPoint, double) method.

Specified by:

setUp in interface UpdateableRegressor

Parameters:

categoricalAttributes - an array containing the categorical attributes that will be in each data point

numericAttributes - the number of numeric attributes that will be in each data point

update

public void update(DataPoint dataPoint,
                   int targetClass)

Description copied from interface: UpdateableClassifier

Updates the classifier by giving it a new data point to learn from.

Specified by:

update in interface UpdateableClassifier

Parameters:

dataPoint - the data point to learn

targetClass - the target class of the data point

update

public void update(DataPoint dataPoint,
                   double targetValue)

Description copied from interface: UpdateableRegressor

Updates the classifier by giving it a new data point to learn from.

Specified by:

update in interface UpdateableRegressor

Parameters:

dataPoint - the data point to learn

targetValue - the target value of the data point

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Specified by:

classify in interface Classifier

Parameters:

data - the data point to classify

Returns:

the results of the classification.

regress

public double regress(DataPoint data)

Specified by:

regress in interface Regressor

train

public void train(ClassificationDataSet dataSet,
                  boolean parallel)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Specified by:

train in interface Classifier

Parameters:

dataSet - the data set to train on

parallel - true if multiple threads should be used to train the model. false if it should be done in a single threaded manner.

train

public void train(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Specified by:

train in interface Classifier

Parameters:

dataSet - the data set to train on

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Specified by:

supportsWeightedData in interface Classifier

Specified by:

supportsWeightedData in interface Regressor

Returns:

true if the model supports weighted data, false otherwise

train

public void train(RegressionDataSet dataSet,
                  boolean parallel)

Specified by:

train in interface Regressor

train

public void train(RegressionDataSet dataSet)

Specified by:

train in interface Regressor

clone

public KernelSGD clone()

Specified by:

clone in interface Classifier

Specified by:

clone in interface UpdateableClassifier

Specified by:

clone in interface Regressor

Specified by:

clone in interface UpdateableRegressor

Overrides:

clone in class java.lang.Object

guessLambda

public static Distribution guessLambda(DataSet d)

Guess the distribution to use for the regularization term λ .

Parameters:

d - the data set to get the guess for

Returns:

the guess for the λ parameter