Catalano.MachineLearning.Classification.DecisionTrees.Learning

Class AdaBoost

java.lang.Object

Catalano.MachineLearning.Classification.DecisionTrees.Learning.AdaBoost

All Implemented Interfaces:

IClassifier, java.io.Serializable, java.lang.Cloneable

public class AdaBoost
extends java.lang.Object
implements IClassifier, java.io.Serializable

AdaBoost (Adaptive Boosting) classifier with decision trees. In principle, AdaBoost is a meta-algorithm, and can be used in conjunction with many other learning algorithms to improve their performance. In practice, AdaBoost with decision trees is probably the most popular combination. AdaBoost is adaptive in the sense that subsequent classifiers built are tweaked in favor of those instances misclassified by previous classifiers. AdaBoost is sensitive to noisy data and outliers. However in some problems it can be less susceptible to the over-fitting problem than most learning algorithms.

AdaBoost calls a weak classifier repeatedly in a series of rounds from total T classifiers. For each call a distribution of weights is updated that indicates the importance of examples in the data set for the classification. On each round, the weights of each incorrectly classified example are increased (or alternatively, the weights of each correctly classified example are decreased), so that the new classifier focuses more on those examples.

The basic AdaBoost algorithm is only for binary classification problem. For multi-class classification, a common approach is reducing the multi-class classification problem to multiple two-class problems. This implementation is a multi-class AdaBoost without such reductions.

References

1. Yoav Freund, Robert E. Schapire. A Decision-Theoretic Generalization of on-Line Learning and an Application to Boosting, 1995.
2. Ji Zhu, Hui Zhou, Saharon Rosset and Trevor Hastie. Multi-class Adaboost, 2009.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
AdaBoost()
AdaBoost(DecisionVariable[] attributes) Initializes a new instance of the AdaBoost class.
AdaBoost(DecisionVariable[] attributes, int T) Initializes a new instance of the AdaBoost class.
AdaBoost(DecisionVariable[] attributes, int T, int J) Initializes a new instance of the AdaBoost class.
AdaBoost(int T) Initializes a new instance of the AdaBoost class.
AdaBoost(int T, int J) Initializes a new instance of the AdaBoost class.

Method Summary

Modifier and Type	Method and Description
IClassifier	clone() Clone of the object.
double[]	getImportance() Returns the variable importance.
int	getNumberOfLeafs() Get the number of maximum leafs.
int	getNumberOfTrees() Get number of trees.
void	Learn(DatasetClassification dataset) Learn from a dataset.
void	Learn(double[][] input, int[] output) Learn.
int	Predict(double[] feature) Predict.
void	setNumberOfLeafs(int J) Set the number of maximum leafs.
void	setNumberOfTrees(int T) Set number of the trees.
int	size() Returns the number of trees in the model.
void	trim(int T) Trims the tree model set to a smaller size in case of over-fitting.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AdaBoost

public AdaBoost()

AdaBoost

public AdaBoost(int T)

Initializes a new instance of the AdaBoost class.

Parameters:

T - the number of trees.

AdaBoost

public AdaBoost(int T,
                int J)

Initializes a new instance of the AdaBoost class.

Parameters:

T - the number of trees.

J - the maximum number of leaf nodes in the trees.

AdaBoost

public AdaBoost(DecisionVariable[] attributes)

Initializes a new instance of the AdaBoost class.

Parameters:

attributes - the attribute properties.

AdaBoost

public AdaBoost(DecisionVariable[] attributes,
                int T)

Initializes a new instance of the AdaBoost class.

Parameters:

attributes - the attribute properties.

T - the number of trees.

AdaBoost

public AdaBoost(DecisionVariable[] attributes,
                int T,
                int J)

Initializes a new instance of the AdaBoost class.

Parameters:

attributes - the attribute properties.

T - the number of trees.

J - the maximum number of leaf nodes in the trees.

Method Detail

getImportance

public double[] getImportance()

Returns the variable importance. Every time a split of a node is made on variable the (GINI, information gain, etc.) impurity criterion for the two descendent nodes is less than the parent node. Adding up the decreases for each individual variable over all trees in the forest gives a simple measure of variable importance.

Returns:

the variable importance

size

public int size()

Returns the number of trees in the model.

Returns:

the number of trees in the model

getNumberOfTrees

public int getNumberOfTrees()

Get number of trees.

Returns:

Number of trees.

setNumberOfTrees

public void setNumberOfTrees(int T)

Set number of the trees.

Parameters:

T - Number of trees.

getNumberOfLeafs

public int getNumberOfLeafs()

Get the number of maximum leafs.

Returns:

Number of maximum leafs.

setNumberOfLeafs

public void setNumberOfLeafs(int J)

Set the number of maximum leafs.

Parameters:

J - Number of maximum leafs.

Learn

public void Learn(DatasetClassification dataset)

Description copied from interface: IClassifier

Learn from a dataset.

Specified by:

Learn in interface IClassifier

Parameters:

dataset - Dataset.

Learn

public void Learn(double[][] input,
                  int[] output)

Description copied from interface: IClassifier

Learn.

Specified by:

Learn in interface IClassifier

Parameters:

input - Matrix of features.

output - Labels.

Predict

public int Predict(double[] feature)

Description copied from interface: IClassifier

Predict.

Specified by:

Predict in interface IClassifier

Parameters:

feature - Feature.

Returns:

Label.

trim

public void trim(int T)

Trims the tree model set to a smaller size in case of over-fitting. Or if extra decision trees in the model don't improve the performance, we may remove them to reduce the model size and also improve the speed of prediction.

Parameters:

T - the new (smaller) size of tree model set.

clone

public IClassifier clone()

Description copied from interface: IClassifier

Clone of the object.

Specified by:

clone in interface IClassifier

Overrides:

clone in class java.lang.Object

Returns:

A new copy of the object.