com.datumbox.framework.core.common.text

Class StringCleaner

java.lang.Object

com.datumbox.framework.core.common.text.StringCleaner

public class StringCleaner
extends java.lang.Object

This is a utility class which contains several methods which are commonly applied to clean-up a string before text classification.

Constructor Summary

Constructors

Constructor and Description
StringCleaner()

Method Summary

Modifier and Type	Method and Description
static java.lang.String	clear(java.lang.String text) Convenience method which tokenizes the URLs and the SMILEYS_MAPPING, removes accents and symbols and eliminates the extra spaces from the provided text.
static java.lang.String	removeAccents(java.lang.String text) Removes all accepts from the text.
static java.lang.String	removeExtraSpaces(java.lang.String text) Replaces all extra spaces with one space.
static java.lang.String	removeSymbols(java.lang.String text) Removes all the non-alphanumeric symbols from the text.
static java.lang.String	tokenizeSmileys(java.lang.String text) Replaces all the SMILEYS_MAPPING within the text with their tokens.
static java.lang.String	tokenizeURLs(java.lang.String text) Replaces all the URLs within the text with a token.
static java.lang.String	unifyTerminators(java.lang.String text) Replaces all terminators with space or dots.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StringCleaner

public StringCleaner()

Method Detail

tokenizeURLs

public static java.lang.String tokenizeURLs(java.lang.String text)

Replaces all the URLs within the text with a token.

Parameters:

text -

Returns:

tokenizeSmileys

public static java.lang.String tokenizeSmileys(java.lang.String text)

Replaces all the SMILEYS_MAPPING within the text with their tokens.

Parameters:

text -

Returns:

removeExtraSpaces

public static java.lang.String removeExtraSpaces(java.lang.String text)

Replaces all extra spaces with one space.

Parameters:

text -

Returns:

removeSymbols

public static java.lang.String removeSymbols(java.lang.String text)

Removes all the non-alphanumeric symbols from the text.

Parameters:

text -

Returns:

unifyTerminators

public static java.lang.String unifyTerminators(java.lang.String text)

Replaces all terminators with space or dots. The final string will contain only alphanumerics and dots.

Parameters:

text -

Returns:

removeAccents

public static java.lang.String removeAccents(java.lang.String text)

Removes all accepts from the text.

Parameters:

text -

Returns:

clear

public static java.lang.String clear(java.lang.String text)

Convenience method which tokenizes the URLs and the SMILEYS_MAPPING, removes accents and symbols and eliminates the extra spaces from the provided text.

Parameters:

text -

Returns: