cc.mallet.pipe

Class FixedVocabTokenizer

java.lang.Object

cc.mallet.pipe.Pipe

cc.mallet.pipe.FixedVocabTokenizer

All Implemented Interfaces:

AlphabetCarrying, java.io.Serializable

public class FixedVocabTokenizer
extends Pipe
implements java.io.Serializable

A simple unicode tokenizer that accepts sequences of letters as tokens.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
int	minimumLength

Constructor Summary

Constructors

Constructor and Description
FixedVocabTokenizer(Alphabet alphabet)

Method Summary

Modifier and Type	Method and Description
Instance	pipe(Instance instance) Really this should be 'protected', but isn't for historical reasons.

Methods inherited from class cc.mallet.pipe.Pipe

alphabetsMatch, getAlphabet, getAlphabets, getDataAlphabet, getInstanceId, getTargetAlphabet, instanceFrom, instancesFrom, instancesFrom, isDataAlphabetSet, isTargetProcessing, newIteratorFrom, precondition, readResolve, setDataAlphabet, setOrCheckDataAlphabet, setOrCheckTargetAlphabet, setTargetAlphabet, setTargetProcessing

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

minimumLength

public int minimumLength

Constructor Detail

FixedVocabTokenizer

public FixedVocabTokenizer(Alphabet alphabet)

Method Detail

pipe

public Instance pipe(Instance instance)

Description copied from class: Pipe

Really this should be 'protected', but isn't for historical reasons.

Overrides:

pipe in class Pipe