cc.mallet.util

Class DBBulkLoader

java.lang.Object

cc.mallet.util.DBBulkLoader

public class DBBulkLoader
extends java.lang.Object

This class reads through two files (data and metadata), tokenizing metadata for use as a label vector.

Constructor Summary

Constructors

Constructor and Description
DBBulkLoader()

Method Summary

Modifier and Type	Method and Description
static void	generateStoplist(SimpleTokenizer prunedTokenizer, NGramPreprocessor preprocessor) Read the data from inputFiles, then write all the words that do not occur pruneCount.value times or more to the pruned word file.
static void	main(java.lang.String[] args)
static void	writeInstanceList(java.util.ArrayList<Pipe> pipes)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DBBulkLoader

public DBBulkLoader()

Method Detail

generateStoplist

public static void generateStoplist(SimpleTokenizer prunedTokenizer,
                                    NGramPreprocessor preprocessor)
                             throws java.io.IOException

Read the data from inputFiles, then write all the words that do not occur pruneCount.value times or more to the pruned word file.

Parameters:

prunedTokenizer - the tokenizer that will be used to write instances

Throws:

java.io.IOException

writeInstanceList

public static void writeInstanceList(java.util.ArrayList<Pipe> pipes)
                              throws java.lang.Exception

Throws:

java.lang.Exception

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception