smile.classification

Class Maxent

java.lang.Object

smile.classification.Maxent

All Implemented Interfaces:

java.io.Serializable, Classifier<int[]>, SoftClassifier<int[]>

public class Maxent
extends java.lang.Object
implements SoftClassifier<int[]>, java.io.Serializable

Maximum Entropy Classifier. Maximum entropy is a technique for learning probability distributions from data. In maximum entropy models, the observed data itself is assumed to be the testable information. Maximum entropy models don't assume anything about the probability distribution other than what have been observed and always choose the most uniform distribution subject to the observed constraints.

Basically, maximum entropy classifier is another name of multinomial logistic regression applied to categorical independent variables, which are converted to binary dummy variables. Maximum entropy models are widely used in natural language processing. Here, we provide an implementation which assumes that binary features are stored in a sparse array, of which entries are the indices of nonzero features.

See Also:

References

1. A. L. Berger, S. D. Pietra, and V. J. D. Pietra. A maximum entropy approach to natural language processing. Computational Linguistics 22(1):39-71, 1996.

, Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Maxent.Trainer Trainer for maximum entropy classifier.

Constructor Summary

Constructors

Constructor and Description
Maxent(int p, int[][] x, int[] y) Learn maximum entropy classifier from samples of binary sparse features.
Maxent(int p, int[][] x, int[] y, double lambda) Learn maximum entropy classifier from samples of binary sparse features.
Maxent(int p, int[][] x, int[] y, double lambda, double tol, int maxIter) Learn maximum entropy classifier from samples of binary sparse features.

Method Summary

Modifier and Type	Method and Description
int	getDimension() Returns the dimension of input space.
double	loglikelihood() Returns the log-likelihood of model.
int	predict(int[] x) Predicts the class label of an instance.
int	predict(int[] x, double[] posteriori) Predicts the class label of an instance and also calculate a posteriori probabilities.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface smile.classification.Classifier

predict

Constructor Detail

Maxent

public Maxent(int p,
              int[][] x,
              int[] y)

Learn maximum entropy classifier from samples of binary sparse features.

Parameters:

p - the dimension of feature space.

x - training samples. Each sample is represented by a set of sparse binary features. The features are stored in an integer array, of which are the indices of nonzero features.

y - training labels in [0, k), where k is the number of classes.

Maxent

public Maxent(int p,
              int[][] x,
              int[] y,
              double lambda)

Learn maximum entropy classifier from samples of binary sparse features.

Parameters:

p - the dimension of feature space.

x - training samples. Each sample is represented by a set of sparse binary features. The features are stored in an integer array, of which are the indices of nonzero features.

y - training labels in [0, k), where k is the number of classes.

lambda - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.

Maxent

public Maxent(int p,
              int[][] x,
              int[] y,
              double lambda,
              double tol,
              int maxIter)

Learn maximum entropy classifier from samples of binary sparse features.

Parameters:

p - the dimension of feature space.

x - training samples. Each sample is represented by a set of sparse binary features. The features are stored in an integer array, of which are the indices of nonzero features.

y - training labels in [0, k), where k is the number of classes.

lambda - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.

tol - tolerance for stopping iterations.

maxIter - maximum number of iterations.

Method Detail

getDimension

public int getDimension()

Returns the dimension of input space.

Returns:

the dimension of input space.

loglikelihood

public double loglikelihood()

Returns the log-likelihood of model.

predict

public int predict(int[] x)

Description copied from interface: Classifier

Predicts the class label of an instance.

Specified by:

predict in interface Classifier<int[]>

Parameters:

x - the instance to be classified.

Returns:

the predicted class label.

predict

public int predict(int[] x,
                   double[] posteriori)

Description copied from interface: SoftClassifier

Predicts the class label of an instance and also calculate a posteriori probabilities. Classifiers may NOT support this method since not all classification algorithms are able to calculate such a posteriori probabilities.

Specified by:

predict in interface SoftClassifier<int[]>

Parameters:

x - the instance to be classified.

posteriori - the array to store a posteriori probabilities on output.

Returns:

the predicted class label