smile.classification

Class LDA

java.lang.Object

smile.classification.LDA

All Implemented Interfaces:

java.io.Serializable, Classifier<double[]>, SoftClassifier<double[]>

public class LDA
extends java.lang.Object
implements SoftClassifier<double[]>, java.io.Serializable

Linear discriminant analysis. LDA is based on the Bayes decision theory and assumes that the conditional probability density functions are normally distributed. LDA also makes the simplifying homoscedastic assumption (i.e. that the class covariances are identical) and that the covariances have full rank. With these assumptions, the discriminant function of an input being in a class is purely a function of this linear combination of independent variables.

LDA is closely related to ANOVA (analysis of variance) and linear regression analysis, which also attempt to express one dependent variable as a linear combination of other features or measurements. In the other two methods, however, the dependent variable is a numerical quantity, while for LDA it is a categorical variable (i.e. the class label). Logistic regression and probit regression are more similar to LDA, as they also explain a categorical variable. These other methods are preferable in applications where it is not reasonable to assume that the independent variables are normally distributed, which is a fundamental assumption of the LDA method.

One complication in applying LDA (and Fisher's discriminant) to real data occurs when the number of variables/features does not exceed the number of samples. In this case, the covariance estimates do not have full rank, and so cannot be inverted. This is known as small sample size problem.

See Also:

FLD, QDA, RDA, NaiveBayes, Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LDA.Trainer Trainer for linear discriminant analysis.

Constructor Summary

Constructors

Constructor and Description
LDA(double[][] x, int[] y) Constructor.
LDA(double[][] x, int[] y, double tol) Constructor.
LDA(double[][] x, int[] y, double[] priori) Constructor.
LDA(double[][] x, int[] y, double[] priori, double tol) Constructor.

Method Summary

Modifier and Type	Method and Description
double[]	getPriori() Returns a priori probabilities.
int	predict(double[] x) Predicts the class label of an instance.
int	predict(double[] x, double[] posteriori) Predicts the class label of an instance and also calculate a posteriori probabilities.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface smile.classification.Classifier

predict

Constructor Detail

LDA

public LDA(double[][] x,
           int[] y)

Constructor. Learn linear discriminant analysis.

Parameters:

x - training samples.

y - training labels in [0, k), where k is the number of classes.

LDA

public LDA(double[][] x,
           int[] y,
           double[] priori)

Constructor. Learn linear discriminant analysis.

Parameters:

x - training samples.

y - training labels in [0, k), where k is the number of classes.

priori - the priori probability of each class.

LDA

public LDA(double[][] x,
           int[] y,
           double tol)

Constructor. Learn linear discriminant analysis.

Parameters:

x - training samples.

y - training labels in [0, k), where k is the number of classes.

tol - a tolerance to decide if a covariance matrix is singular; it will reject variables whose variance is less than tol².

LDA

public LDA(double[][] x,
           int[] y,
           double[] priori,
           double tol)

Constructor. Learn linear discriminant analysis.

Parameters:

x - training samples.

y - training labels in [0, k), where k is the number of classes.

priori - the priori probability of each class. If null, it will be estimated from the training data.

tol - a tolerance to decide if a covariance matrix is singular; it will reject variables whose variance is less than tol².

Method Detail

getPriori

public double[] getPriori()

Returns a priori probabilities.

predict

public int predict(double[] x)

Description copied from interface: Classifier

Predicts the class label of an instance.

Specified by:

predict in interface Classifier<double[]>

Parameters:

x - the instance to be classified.

Returns:

the predicted class label.

predict

public int predict(double[] x,
                   double[] posteriori)

Description copied from interface: SoftClassifier

Predicts the class label of an instance and also calculate a posteriori probabilities. Classifiers may NOT support this method since not all classification algorithms are able to calculate such a posteriori probabilities.

Specified by:

predict in interface SoftClassifier<double[]>

Parameters:

x - the instance to be classified.

posteriori - the array to store a posteriori probabilities on output.

Returns:

the predicted class label