Documentation of 'jsat.text.topicmodel.OnlineLDAsvi' Java class
OnlineLDAsvi
Skip navigation links
jsat.text.topicmodel

Class OnlineLDAsvi

  • All Implemented Interfaces:
    Parameterized

    public class OnlineLDAsvi
extends java.lang.Object
implements Parameterized
    This class provides an implementation of Latent Dirichlet Allocation for learning a topic model from a set of documents. This implementation is based on Stochastic Variational Inference and is meant for large collections (more than 100,000 data points) and can learn in an online fashion.

    For LDA it is common to set α = η = 1/K, where K is the number of topics to be learned. Note that η is not a learning rate parameter, as the symbol is usually used.
    For this algorithm, some potential parameter combinations (by column) for batch size, κ, and τ0 are:
    batch size 256 1024 4096
    κ 0.6 0.5 0.5
    τ0 1024 256 64

    For smaller corpuses, reducing τ0 can improve the performance (even down to τ0 = 1)
    See:
    • Blei, D. M., Ng, A. Y.,&Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3(4-5), 993–1022. doi:10.1162/jmlr.2003.3.4-5.993
    • Hoffman, M., Blei, D.,&Bach, F. (2010). Online Learning for Latent Dirichlet Allocation. In Advances in Neural Information Processing Systems (pp. 856–864). Retrieved from here
    • Hoffman, M. D., Blei, D. M., Wang, C.,&Paisley, J. (2013). Stochastic Variational Inference. The Journal of Machine Learning Research, 14(1), 1303–1347.
    • Hoffman, M. D. (2013). Lazy updates for online LDA. Retrieved from here

    • Constructor Summary

      Constructors 
      Constructor and Description
      OnlineLDAsvi()
      Creates a new Online LDA learner.
      OnlineLDAsvi(int K, int D, int W)
      Creates a new Online LDA learner that is ready for online updates

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method and Description
      double getAlpha() 
      int getD()
      Returns the approximate number of documents that will be observed, or -1 if this object is not ready to learn
      int getEpochs()
      Returns the number of training iterations over the data set that will be used
      double getEta() 
      int getK()
      Returns the number of topics to learn, or -1 if this object is not ready to learn
      double getKappa() 
      Vec getTopics(Vec doc)
      Computes the topic distribution for the given document.
      Note that the returned vector will be dense, but many of the values may be very nearly zero.
      Vec getTopicVec(int k)
      Returns the topic vector for a given topic.
      int getVocabSize()
      Returns the size of the vocabulary for LDA, or -1 if this object is not ready to learn
      void model(DataSet dataSet, int topics)
      Fits the LDA model against the given data set
      void model(DataSet dataSet, int topics, java.util.concurrent.ExecutorService ex)
      Fits the LDA model against the given data set
      void setAlpha(double alpha)
      Sets the prior for the on weight vector theta.
      void setD(int D)
      Sets the approximate number of documents that will be observed
      void setEpochs(int epochs)
      Sets the number of training epochs when learning in a "batch" setting
      void setEta(double eta)
      Prior on topics.
      void setK(int K)
      Sets the number of topics that LDA will try to learn
      void setKappa(double kappa)
      The "forgetfulness" factor in the learning rate.
      void setMiniBatchSize(int miniBatchSize)
      Sets the number of data points used at a time to perform one update of the model parameters
      void setTau0(double tau0)
      A learning rate constant to control the influence of early iterations on the solution.
      void setVocabSize(int W)
      Sets the vocabulary size for LDA, which is the number of dimensions in the input feature vectors.
      void update(java.util.List<Vec> docs)
      Performs an update of the LDA topic distributions based on the given mini-batch of documents.
      void update(java.util.List<Vec> docs, java.util.concurrent.ExecutorService ex)
      Performs an update of the LDA topic distribution based on the given mini-batch of documents.

      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • OnlineLDAsvi

        public OnlineLDAsvi()
        Creates a new Online LDA learner. The number of topics, expected number of documents, and the vocabulary size must be set before it can be used.

      • OnlineLDAsvi

        public OnlineLDAsvi(int K,
                    int D,
                    int W)
        Creates a new Online LDA learner that is ready for online updates
        Parameters:
        K - the number of topics to learn
        D - the expected number of documents to see
        W - the vocabulary size

    • Method Detail

      • setK

        public void setK(int K)
        Sets the number of topics that LDA will try to learn
        Parameters:
        K - the number of topics to learn

      • getK

        public int getK()
        Returns the number of topics to learn, or -1 if this object is not ready to learn
        Returns:
        the number of topics that will be learned

      • setD

        public void setD(int D)
        Sets the approximate number of documents that will be observed
        Parameters:
        D - the number of documents that will be observed

      • getD

        public int getD()
        Returns the approximate number of documents that will be observed, or -1 if this object is not ready to learn
        Returns:
        the number of documents that will be observed

      • setVocabSize

        public void setVocabSize(int W)
        Sets the vocabulary size for LDA, which is the number of dimensions in the input feature vectors.
        Parameters:
        W - the vocabulary size for LDA

      • getVocabSize

        public int getVocabSize()
        Returns the size of the vocabulary for LDA, or -1 if this object is not ready to learn
        Returns:
        the vocabulary size for LDA

      • setAlpha

        public void setAlpha(double alpha)
        Sets the prior for the on weight vector theta. 1/K is a common choice.
        Parameters:
        alpha - the positive prior value

      • getAlpha

        public double getAlpha()
        Returns:
        the weight vector prior over theta

      • setEta

        public void setEta(double eta)
        Prior on topics. 1/K is a common choice.
        Parameters:
        eta - the positive prior for topics

      • getEta

        public double getEta()
        Returns:
        the topic prior

      • setTau0

        public void setTau0(double tau0)
        A learning rate constant to control the influence of early iterations on the solution. Larger values reduce the influence of earlier iterations, smaller values increase the weight of earlier iterations.
        Parameters:
        tau0 - a learning rate parameter that must be greater than 0 (usually at least 1)

      • setEpochs

        public void setEpochs(int epochs)
        Sets the number of training epochs when learning in a "batch" setting
        Parameters:
        epochs - the number of iterations to go over the data set

      • getEpochs

        public int getEpochs()
        Returns the number of training iterations over the data set that will be used
        Returns:
        the number of training iterations over the data set that will be used

      • setKappa

        public void setKappa(double kappa)
        The "forgetfulness" factor in the learning rate. Larger values increase the rate at which old information is "forgotten"
        Parameters:
        kappa - the forgetfulness factor in [0.5, 1]

      • getKappa

        public double getKappa()
        Returns:
        the forgetfulness factor

      • setMiniBatchSize

        public void setMiniBatchSize(int miniBatchSize)
        Sets the number of data points used at a time to perform one update of the model parameters
        Parameters:
        miniBatchSize - the batch size to use

      • getTopicVec

        public Vec getTopicVec(int k)
        Returns the topic vector for a given topic. The vector should not be altered, and is scaled so that the sum of all term weights sums to one.
        Parameters:
        k - the topic to get the vector for
        Returns:
        the raw topic vector for the requested topic.

      • update

        public void update(java.util.List<Vec> docs)
        Performs an update of the LDA topic distributions based on the given mini-batch of documents.
        Parameters:
        docs - the list of document vectors to update from

      • update

        public void update(java.util.List<Vec> docs,
                   java.util.concurrent.ExecutorService ex)
        Performs an update of the LDA topic distribution based on the given mini-batch of documents.
        Parameters:
        docs - the list of document vectors to update from
        ex - the source of threads for parallel execution

      • model

        public void model(DataSet dataSet,
                  int topics)
        Fits the LDA model against the given data set
        Parameters:
        dataSet - the data set to learn a topic model for
        topics - the number of topics to learn

      • model

        public void model(DataSet dataSet,
                  int topics,
                  java.util.concurrent.ExecutorService ex)
        Fits the LDA model against the given data set
        Parameters:
        dataSet - the data set to learn a topic model for
        topics - the number of topics to learn
        ex - the source of threads for parallel execution

      • getTopics

        public Vec getTopics(Vec doc)
        Computes the topic distribution for the given document.
        Note that the returned vector will be dense, but many of the values may be very nearly zero.
        Parameters:
        doc - the document to find the topics for
        Returns:
        a vector of the topic distribution for the given document
Skip navigation links

DataMelt 3.0 © DataMelt by jWork.ORG

You see the box below because you did not login.