jsat.text

Class BasicTextVectorCreator

java.lang.Object

jsat.text.BasicTextVectorCreator

All Implemented Interfaces:

java.io.Serializable, TextVectorCreator

public class BasicTextVectorCreator
extends java.lang.Object
implements TextVectorCreator

Creates new text vectors from a dictionary of known tokens and a word weighting scheme.

This object is generally intended to be constructed by a TextDataLoader, though can be used if you know all the words you will need (and can initialize the WordWeighting) before creating this object.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
BasicTextVectorCreator(Tokenizer tokenizer, java.util.Map<java.lang.String,java.lang.Integer> wordIndex, WordWeighting weighting) Creates a new basic text vector creator

Method Summary

Modifier and Type	Method and Description
Vec	newText(java.lang.String text) Converts the given input text into a vector representation.
Vec	newText(java.lang.String input, java.lang.StringBuilder workSpace, java.util.List<java.lang.String> storageSpace) Converts the given input text into a vector representation

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BasicTextVectorCreator

public BasicTextVectorCreator(Tokenizer tokenizer,
                              java.util.Map<java.lang.String,java.lang.Integer> wordIndex,
                              WordWeighting weighting)

Creates a new basic text vector creator

Parameters:

tokenizer - the tokenizer to apply to incoming strings

wordIndex - the map of each known word to its index, the size of the map indicating the maximum (exclusive) index

weighting - the weighting process to apply to each loaded document. This should have already been initialized, or be stateless.

Method Detail

newText

public Vec newText(java.lang.String text)

Description copied from interface: TextVectorCreator

Converts the given input text into a vector representation.

Specified by:

newText in interface TextVectorCreator

Parameters:

text - the input string

Returns:

a vector representation

newText

public Vec newText(java.lang.String input,
                   java.lang.StringBuilder workSpace,
                   java.util.List<java.lang.String> storageSpace)

Description copied from interface: TextVectorCreator

Converts the given input text into a vector representation

Specified by:

newText in interface TextVectorCreator

Parameters:

input - the input string

workSpace - an already allocated (but empty) string builder than can be used as a temporary work space.

storageSpace - an already allocated (but empty) list to place the tokens into

Returns:

a vector representation