jsat.datatransform

Class AutoDeskewTransform

java.lang.Object

jsat.datatransform.AutoDeskewTransform

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, DataTransform, InPlaceTransform

public class AutoDeskewTransform
extends java.lang.Object
implements InPlaceTransform

This transform applies a shifted Box-Cox transform for several fixed values of λ, and selects the one that provides the greatest reduction in the skewness of the distribution. This is done in an attempt to make the individual features appear more normal. The shifted values are done to preserve zeros and keep sparse inputs sparse. This is done with two passes through the data set, but requires only O(D #λ values) memory.

The default values of λ are -1, -1/2, 0, 1/2, 1. When using negative λ values all zeros are skipped and left as zeros. λ = 1 is an implicit value that is always included regardless of the input, as it is equivalent to leaving the data unchanged when preserving zero values. The stated default values include the log(x+1) and sqrt(x) transforms that are commonly used for deskewing as special cases.

Skewness can be calculated by including zero, but by default ignores them as "not-present" values.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
AutoDeskewTransform() Creates a new AutoDeskew transform
AutoDeskewTransform(boolean ignorZeros, java.util.List<java.lang.Double> lambdas) Creates a new AutoDeskew transform
AutoDeskewTransform(DataSet dataSet) Creates a new deskewing object from the given data set
AutoDeskewTransform(DataSet dataSet, boolean ignorZeros, java.util.List<java.lang.Double> lambdas) Creates a new deskewing object from the given data set
AutoDeskewTransform(DataSet dataSet, java.util.List<java.lang.Double> lambdas) Creates a new deskewing object from the given data set
AutoDeskewTransform(double... lambdas) Creates a new AutoDeskew transform
AutoDeskewTransform(java.util.List<java.lang.Double> lambdas) Creates a new AutoDeskew transform

Method Summary

Modifier and Type	Method and Description
AutoDeskewTransform	clone()
void	fit(DataSet dataSet) Fits this transform to the given dataset.
void	mutableTransform(DataPoint dp) Mutates the given data point.
boolean	mutatesNominal() By default returns false.
DataPoint	transform(DataPoint dp) Returns a new data point that is a transformation of the original data point.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AutoDeskewTransform

public AutoDeskewTransform()

Creates a new AutoDeskew transform

AutoDeskewTransform

public AutoDeskewTransform(double... lambdas)

Creates a new AutoDeskew transform

Parameters:

lambdas - the list of lambda values to evaluate

AutoDeskewTransform

public AutoDeskewTransform(java.util.List<java.lang.Double> lambdas)

Creates a new AutoDeskew transform

Parameters:

lambdas - the list of lambda values to evaluate

AutoDeskewTransform

public AutoDeskewTransform(boolean ignorZeros,
                           java.util.List<java.lang.Double> lambdas)

Creates a new AutoDeskew transform

Parameters:

ignorZeros - true to ignore zero values when calculating the skewness, false to include them.

lambdas - the list of lambda values to evaluate

AutoDeskewTransform

public AutoDeskewTransform(DataSet dataSet)

Creates a new deskewing object from the given data set

Parameters:

dataSet - the data set to deskew

AutoDeskewTransform

public AutoDeskewTransform(DataSet dataSet,
                           java.util.List<java.lang.Double> lambdas)

Creates a new deskewing object from the given data set

Parameters:

dataSet - the data set to deskew

lambdas - the list of lambda values to evaluate

AutoDeskewTransform

public AutoDeskewTransform(DataSet dataSet,
                           boolean ignorZeros,
                           java.util.List<java.lang.Double> lambdas)

Creates a new deskewing object from the given data set

Parameters:

dataSet - the data set to deskew

ignorZeros - true to ignore zero values when calculating the skewness, false to include them.

lambdas - the list of lambda values to evaluate

Method Detail

fit

public void fit(DataSet dataSet)

Description copied from interface: DataTransform

Fits this transform to the given dataset. Some transforms can only be learned from classification or regression datasets. If an incompatible dataset type is given, a FailedToFitException exception may be thrown.

Specified by:

fit in interface DataTransform

Parameters:

dataSet - the dataset to fir this transform to

transform

public DataPoint transform(DataPoint dp)

Description copied from interface: DataTransform

Returns a new data point that is a transformation of the original data point. This new data point is a different object, but may contain the same references as the original data point. It is not guaranteed that you can mutate the transformed point without having a side effect on the original point.

Specified by:

transform in interface DataTransform

Parameters:

dp - the data point to apply a transformation to

Returns:

a transformed data point

mutableTransform

public void mutableTransform(DataPoint dp)

Description copied from interface: InPlaceTransform

Mutates the given data point. This causes side effects, altering the data point to have the same value as the output of DataTransform.transform(jsat.classifiers.DataPoint).

Specified by:

mutableTransform in interface InPlaceTransform

Parameters:

dp - the data point to alter

clone

public AutoDeskewTransform clone()

Specified by:

clone in interface DataTransform

Overrides:

clone in class java.lang.Object

mutatesNominal

public boolean mutatesNominal()

Description copied from interface: InPlaceTransform

By default returns false. Only returns true if this transform will mutableTransform the nominal feature values of a data point.

Specified by:

mutatesNominal in interface InPlaceTransform

Returns:

true if nominal feature values are mutated, false otherwise.