cc.mallet.pipe

Class CharSequence2TokenSequence

java.lang.Object

cc.mallet.pipe.Pipe

cc.mallet.pipe.CharSequence2TokenSequence

All Implemented Interfaces:

AlphabetCarrying, java.io.Serializable

public class CharSequence2TokenSequence
extends Pipe
implements java.io.Serializable

Pipe that tokenizes a character sequence. Expects a CharSequence in the Instance data, and converts the sequence into a token sequence using the given regex or CharSequenceLexer. (The regex / lexer should specify what counts as a token.)

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
CharSequence2TokenSequence()
CharSequence2TokenSequence(CharSequenceLexer lexer)
CharSequence2TokenSequence(java.util.regex.Pattern regex)
CharSequence2TokenSequence(java.lang.String regex)

Method Summary

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)
Instance	pipe(Instance carrier) Really this should be 'protected', but isn't for historical reasons.

Methods inherited from class cc.mallet.pipe.Pipe

alphabetsMatch, getAlphabet, getAlphabets, getDataAlphabet, getInstanceId, getTargetAlphabet, instanceFrom, instancesFrom, instancesFrom, isDataAlphabetSet, isTargetProcessing, newIteratorFrom, precondition, readResolve, setDataAlphabet, setOrCheckDataAlphabet, setOrCheckTargetAlphabet, setTargetAlphabet, setTargetProcessing

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CharSequence2TokenSequence

public CharSequence2TokenSequence(CharSequenceLexer lexer)

CharSequence2TokenSequence

public CharSequence2TokenSequence(java.lang.String regex)

CharSequence2TokenSequence

public CharSequence2TokenSequence(java.util.regex.Pattern regex)

CharSequence2TokenSequence

public CharSequence2TokenSequence()

Method Detail

pipe

public Instance pipe(Instance carrier)

Description copied from class: Pipe

Really this should be 'protected', but isn't for historical reasons.

Overrides:

pipe in class Pipe

main

public static void main(java.lang.String[] args)