smile.classification

Class LogisticRegression

java.lang.Object

smile.classification.LogisticRegression

All Implemented Interfaces:

java.io.Serializable, Classifier<double[]>, SoftClassifier<double[]>

public class LogisticRegression
extends java.lang.Object
implements SoftClassifier<double[]>, java.io.Serializable

Logistic regression. Logistic regression (logit model) is a generalized linear model used for binomial regression. Logistic regression applies maximum likelihood estimation after transforming the dependent into a logit variable. A logit is the natural log of the odds of the dependent equaling a certain value or not (usually 1 in binary logistic models, the highest value in multinomial models). In this way, logistic regression estimates the odds of a certain event (value) occurring.

Goodness-of-fit tests such as the likelihood ratio test are available as indicators of model appropriateness, as is the Wald statistic to test the significance of individual independent variables.

Logistic regression has many analogies to ordinary least squares (OLS) regression. Unlike OLS regression, however, logistic regression does not assume linearity of relationship between the raw values of the independent variables and the dependent, does not require normally distributed variables, does not assume homoscedasticity, and in general has less stringent requirements.

Compared with linear discriminant analysis, logistic regression has several advantages:

• It is more robust: the independent variables don't have to be normally distributed, or have equal variance in each group
• It does not assume a linear relationship between the independent variables and dependent variable.
• It may handle nonlinear effects since one can add explicit interaction and power terms.

However, it requires much more data to achieve stable, meaningful results.

Logistic regression also has strong connections with neural network and maximum entropy modeling. For example, binary logistic regression is equivalent to a one-layer, single-output neural network with a logistic activation function trained under log loss. Similarly, multinomial logistic regression is equivalent to a one-layer, softmax-output neural network.

Logistic regression estimation also obeys the maximum entropy principle, and thus logistic regression is sometimes called "maximum entropy modeling", and the resulting classifier the "maximum entropy classifier".

See Also:

NeuralNetwork, Maxent, LDA, Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LogisticRegression.Trainer Trainer for logistic regression.

Constructor Summary

Constructors

Constructor and Description
LogisticRegression(double[][] x, int[] y) Constructor.
LogisticRegression(double[][] x, int[] y, double lambda) Constructor.
LogisticRegression(double[][] x, int[] y, double lambda, double tol, int maxIter) Constructor.

Method Summary

Modifier and Type	Method and Description
double	loglikelihood() Returns the log-likelihood of model.
int	predict(double[] x) Predicts the class label of an instance.
int	predict(double[] x, double[] posteriori) Predicts the class label of an instance and also calculate a posteriori probabilities.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface smile.classification.Classifier

predict

Constructor Detail

LogisticRegression

public LogisticRegression(double[][] x,
                          int[] y)

Constructor. No regularization.

Parameters:

x - training samples.

y - training labels in [0, k), where k is the number of classes.

LogisticRegression

public LogisticRegression(double[][] x,
                          int[] y,
                          double lambda)

Constructor.

Parameters:

x - training samples.

y - training labels in [0, k), where k is the number of classes.

lambda - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.

LogisticRegression

public LogisticRegression(double[][] x,
                          int[] y,
                          double lambda,
                          double tol,
                          int maxIter)

Constructor.

Parameters:

x - training samples.

y - training labels in [0, k), where k is the number of classes.

lambda - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.

tol - the tolerance for stopping iterations.

maxIter - the maximum number of iterations.

Method Detail

loglikelihood

public double loglikelihood()

Returns the log-likelihood of model.

predict

public int predict(double[] x)

Description copied from interface: Classifier

Predicts the class label of an instance.

Specified by:

predict in interface Classifier<double[]>

Parameters:

x - the instance to be classified.

Returns:

the predicted class label.

predict

public int predict(double[] x,
                   double[] posteriori)

Description copied from interface: SoftClassifier

Predicts the class label of an instance and also calculate a posteriori probabilities. Classifiers may NOT support this method since not all classification algorithms are able to calculate such a posteriori probabilities.

Specified by:

predict in interface SoftClassifier<double[]>

Parameters:

x - the instance to be classified.

posteriori - the array to store a posteriori probabilities on output.

Returns:

the predicted class label