jsat.text.wordweighting

Interface WordWeighting

All Superinterfaces:

Function, IndexFunction, java.io.Serializable

All Known Implementing Classes:

BinaryWordPresent, OkapiBM25, TfIdf, WordCount

public interface WordWeighting
extends IndexFunction

WordWeighting is an index function specifically mean for modifying the values of a vectors used for a bag-of-words representation of text data.
Some Word weighting schemes may need information about the document collection as a whole before constructing the weightings, and this class provides the facilities for this to be done in a standardized manner.

Method Summary

Modifier and Type	Method and Description
void	applyTo(Vec vec) The implementation may want to pre compute come values based on the vector it is about to be applied to.
void	setWeight(java.util.List<? extends Vec> allDocuments, java.util.List<java.lang.Integer> df) Prepares the word weighting to be performed on a data set.

Methods inherited from interface jsat.math.IndexFunction

f, indexFunc

Methods inherited from interface jsat.math.Function

f, f, forwardDifference

Method Detail

setWeight

void setWeight(java.util.List<? extends Vec> allDocuments,
               java.util.List<java.lang.Integer> df)

Prepares the word weighting to be performed on a data set. This should be called once before being applied to any vectors. Different WordWeightings may require different amounts of computation to set up.

Parameters:

allDocuments - the list of all vectors that make up the set of documents. The word vectors should be unmodified, containing the value of how many times a word appeared in the document for each index.

df - a list mapping each integer index of a word to how many times that word occurred in total

applyTo

void applyTo(Vec vec)

The implementation may want to pre compute come values based on the vector it is about to be applied to. This should be called in place of Vec.applyIndexFunction(jsat.math.IndexFunction) . The vector should be in a bag-of-words form where each index value indicates how many times the word for that index occurred in the document represented by the vector.

Parameters:

vec - the vector to set up for and then alter by invoking Vec.applyIndexFunction(jsat.math.IndexFunction) on