com.datumbox.framework.core.common.text.tokenizers

Class WhitespaceTokenizer

java.lang.Object

com.datumbox.framework.core.common.text.tokenizers.AbstractTokenizer

com.datumbox.framework.core.common.text.tokenizers.WhitespaceTokenizer

public class WhitespaceTokenizer
extends AbstractTokenizer

The white space tokenizer separates the keywords of a string using the white space.

Constructor Summary

Constructors

Constructor and Description
WhitespaceTokenizer()

Method Summary

Modifier and Type	Method and Description
java.util.List<java.lang.String>	tokenize(java.lang.String text) Separates the tokens of a string by splitting it on white space.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WhitespaceTokenizer

public WhitespaceTokenizer()

Method Detail

tokenize

public java.util.List<java.lang.String> tokenize(java.lang.String text)

Separates the tokens of a string by splitting it on white space.

Specified by:

tokenize in class AbstractTokenizer

Parameters:

text -

Returns: