org.apache.lucene.analysis

Class SinkTokenizer

java.lang.Object

org.apache.lucene.analysis.TokenStream

org.apache.lucene.analysis.Tokenizer

org.apache.lucene.analysis.SinkTokenizer

public class SinkTokenizer
extends Tokenizer

A SinkTokenizer can be used to cache Tokens for use in an Analyzer

See Also:

TeeTokenFilter

Constructor Summary

Constructors

Constructor and Description
SinkTokenizer()
SinkTokenizer(int initCap)
SinkTokenizer(java.util.List input)

Method Summary

Modifier and Type	Method and Description
void	add(Token t) Override this method to cache only certain tokens, or new tokens based on the old tokens.
void	close() By default, closes the input Reader.
java.util.List	getTokens() Get the tokens in the internal List.
Token	next() Returns the next token out of the list of cached tokens
void	reset() Reset the internal data structures to the start at the front of the list of tokens.

Methods inherited from class org.apache.lucene.analysis.Tokenizer

reset

Methods inherited from class org.apache.lucene.analysis.TokenStream

next

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SinkTokenizer

public SinkTokenizer(java.util.List input)

SinkTokenizer

public SinkTokenizer()

SinkTokenizer

public SinkTokenizer(int initCap)

Method Detail

getTokens

public java.util.List getTokens()

Get the tokens in the internal List.

WARNING: Adding tokens to this list requires the reset() method to be called in order for them to be made available. Also, this Tokenizer does nothing to protect against ConcurrentModificationExceptions in the case of adds happening while TokenStream.next(org.apache.lucene.analysis.Token) is being called.

Returns:

A List of Tokens

next

public Token next()
           throws java.io.IOException

Returns the next token out of the list of cached tokens

Overrides:

next in class TokenStream

Returns:

The next Token in the Sink.

Throws:

java.io.IOException

add

public void add(Token t)

Override this method to cache only certain tokens, or new tokens based on the old tokens.

Parameters:

t - The Token to add to the sink

close

public void close()
           throws java.io.IOException

Description copied from class: Tokenizer

By default, closes the input Reader.

Overrides:

close in class Tokenizer

Throws:

java.io.IOException

reset

public void reset()
           throws java.io.IOException

Reset the internal data structures to the start at the front of the list of tokens. Should be called if tokens were added to the list after an invocation of TokenStream.next(Token)

Overrides:

reset in class TokenStream

Throws:

java.io.IOException