Documentation of 'org.python.indexer.Indexer' Java class
Indexer
Skip navigation links
org.python.indexer

Class Indexer


  • public class Indexer
extends java.lang.Object
    Indexes a set of Python files and builds a code graph.

    This class is not thread-safe.

    • Field Summary

      Fields 
      Modifier and Type Field and Description
      Builtins builtins
      Manages the built-in modules -- that is, modules from the standard Python library that are implemented in C and consequently have no Python source.
      java.lang.String currentFile 
      java.util.Set<java.lang.String> failedModules
      When resolving imports we look in various possible locations.
      Scope globaltable
      The top-level (builtin) scope.
      static Indexer idx
      The global indexer instance.
      Scope moduleTable
      A scope containing bindings for all modules currently loaded by the indexer.
      java.util.Map<java.lang.String,java.util.List<Diagnostic>> parseErrs 
      java.util.List<java.lang.String> path 
      java.util.Map<java.lang.String,java.util.List<Diagnostic>> problems
      Diagnostics.
      java.lang.String projDir 

    • Constructor Summary

      Constructors 
      Constructor and Description
      Indexer() 

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method and Description
      void addPath(java.lang.String p)
      Adds the specified absolute path to the module search path.
      void addPaths(java.util.List<java.lang.String> p)
      Adds the specified absolute paths to the module search path.
      boolean aggressiveAssertionsEnabled() 
      void clearAstCache()
      Clears the AST cache (to free up memory).
      void clearModuleTable()
      Clears the module table, discarding all resolved ASTs (modules) and their scope information.
      void enableAggressiveAssertions(boolean enable)
      Configures whether the indexer should abort with an exception when it encounters an internal error or unexpected program state.
      void fine(java.lang.String msg) 
      void finer(java.lang.String msg) 
      java.util.List<Outliner.Entry> generateOutline(java.lang.String path)
      Create an outline for a file in the index.
      NModule getAstForFile(java.lang.String file)
      Returns the syntax tree for file.
      NModule getAstForFile(java.lang.String file, java.lang.String contents)
      Returns the syntax tree for file.
      java.util.Map<java.lang.String,NBinding> getBindings()
      Returns the mutable set of all bindings collected, keyed on their qnames.
      NModuleType getBuiltinModule(java.lang.String qname) 
      java.util.List<Diagnostic> getDiagnosticsForFile(java.lang.String file)
      Returns the list, possibly empty but never null, of errors and warnings generated in the file.
      java.util.List<java.lang.String> getLoadedFiles() 
      java.util.List<java.lang.String> getLoadPath()
      Returns the module search path -- the project directory followed by any paths that were added by addPath(String).
      java.util.logging.Logger getLogger() 
      NModuleType getModuleForFile(java.lang.String file)
      Returns (loading/resolving if necessary) the module for a given source path.
      java.lang.String getStatusReport()
      Report resolution rate and other statistics data.
      void handleException(java.lang.String msg, java.lang.Throwable cause)
      If aggressive assertions are enabled, propagates the passed Throwable, wrapped in an IndexingException.
      void info(java.lang.String msg) 
      boolean isLibFile(java.lang.String file) 
      NModuleType loadFile(java.lang.String path)
      Loads a file and all its ancestor packages.
      NModuleType loadFile(java.lang.String path, boolean skipChain)
      Load, parse and analyze a source file given its absolute path.
      void loadFileRecursive(java.lang.String fullname)
      Load all Python source files recursively if the given fullname is a directory; otherwise just load a file.
      NModuleType loadModule(java.lang.String modname)
      This method searches the module path for the module modname.
      NModuleType loadString(java.lang.String path, java.lang.String contents)
      Loads a module from a string containing the module contents.
      void log(java.util.logging.Level level, java.lang.String msg) 
      NBinding lookupQname(java.lang.String qname)
      Return the binding for qname, or null if not known.
      NType lookupQnameType(java.lang.String qname)
      Return the type for qname, or null if not known.
      int numFilesLoaded() 
      NBinding putBinding(NBinding b) 
      void putLocation(NNode node, NBinding b)
      Add a reference to binding b at AST node node.
      void putLocation(Ref ref, NBinding b) 
      void putProblem(NNode loc, java.lang.String msg) 
      void putProblem(java.lang.String file, int beg, int end, java.lang.String msg) 
      void ready()
      Performs final indexing-building passes, including marking references to undeclared variables.
      void recordUnresolvedModule(java.lang.String qname, java.lang.String file)
      Reports a failed module or submodule resolution.
      void release()
      Releases all resources for the current indexer.
      void removeBinding(NBinding b) 
      void reportFailedAssertion(java.lang.String msg)
      Signals a failed assertion about the state of the indexer or index.
      void setLogger(java.util.logging.Logger logger) 
      void setPath(java.util.List<java.lang.String> path)
      Sets the module search path to the specified list of absolute paths.
      void setProjectDir(java.lang.String cd) 
      void severe(java.lang.String msg) 
      java.lang.String toString() 
      void updateLocation(Ref node, NBinding b)
      Add node as a reference to binding b, removing node from any other binding ref-lists that it may have occupied.
      void warn(java.lang.String msg) 

      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Field Detail

      • idx

        public static Indexer idx
        The global indexer instance. Provides convenient access to global resources, as well as easy cleanup of resources after the index is built.

      • moduleTable

        public Scope moduleTable
        A scope containing bindings for all modules currently loaded by the indexer.

      • globaltable

        public Scope globaltable
        The top-level (builtin) scope.

      • problems

        public java.util.Map<java.lang.String,java.util.List<Diagnostic>> problems
        Diagnostics.

      • parseErrs

        public java.util.Map<java.lang.String,java.util.List<Diagnostic>> parseErrs

      • currentFile

        public java.lang.String currentFile

      • projDir

        public java.lang.String projDir

      • path

        public java.util.List<java.lang.String> path

      • failedModules

        public java.util.Set<java.lang.String> failedModules
        When resolving imports we look in various possible locations. This set keeps track of modules we attempted but didn't find.

      • builtins

        public Builtins builtins
        Manages the built-in modules -- that is, modules from the standard Python library that are implemented in C and consequently have no Python source.

    • Constructor Detail

      • Indexer

        public Indexer()

    • Method Detail

      • setLogger

        public void setLogger(java.util.logging.Logger logger)

      • getLogger

        public java.util.logging.Logger getLogger()

      • setProjectDir

        public void setProjectDir(java.lang.String cd)
                   throws java.io.IOException
        Throws:
        java.io.IOException

      • enableAggressiveAssertions

        public void enableAggressiveAssertions(boolean enable)
        Configures whether the indexer should abort with an exception when it encounters an internal error or unexpected program state. Normally the indexer attempts to continue indexing, on the assumption that having an index with mostly good data is better than having no index at all. Enabling aggressive assertions is useful for debugging the indexer.

      • aggressiveAssertionsEnabled

        public boolean aggressiveAssertionsEnabled()

      • handleException

        public void handleException(java.lang.String msg,
                            java.lang.Throwable cause)
        If aggressive assertions are enabled, propagates the passed Throwable, wrapped in an IndexingException.
        Parameters:
        msg - descriptive message; ok to be null
        Throws:
        IndexingException

      • reportFailedAssertion

        public void reportFailedAssertion(java.lang.String msg)
        Signals a failed assertion about the state of the indexer or index. If aggressive assertions are enabled, throws an IndexingException. Otherwise the message is logged as a warning, and indexing continues.
        Parameters:
        msg - a descriptive message about the problem
        Throws:
        IndexingException
        See Also:
        enableAggressiveAssertions(boolean)

      • addPaths

        public void addPaths(java.util.List<java.lang.String> p)
              throws java.io.IOException
        Adds the specified absolute paths to the module search path.
        Throws:
        java.io.IOException

      • addPath

        public void addPath(java.lang.String p)
             throws java.io.IOException
        Adds the specified absolute path to the module search path.
        Throws:
        java.io.IOException

      • setPath

        public void setPath(java.util.List<java.lang.String> path)
             throws java.io.IOException
        Sets the module search path to the specified list of absolute paths.
        Throws:
        java.io.IOException

      • getLoadPath

        public java.util.List<java.lang.String> getLoadPath()
        Returns the module search path -- the project directory followed by any paths that were added by addPath(String).

      • isLibFile

        public boolean isLibFile(java.lang.String file)

      • getBindings

        public java.util.Map<java.lang.String,NBinding> getBindings()
        Returns the mutable set of all bindings collected, keyed on their qnames.

      • lookupQname

        public NBinding lookupQname(java.lang.String qname)
        Return the binding for qname, or null if not known.

      • lookupQnameType

        public NType lookupQnameType(java.lang.String qname)
        Return the type for qname, or null if not known.
        Throws:
        java.lang.IllegalStateException - if ready() has not been called.

      • getModuleForFile

        public NModuleType getModuleForFile(java.lang.String file)
                             throws java.lang.Exception
        Returns (loading/resolving if necessary) the module for a given source path.
        Parameters:
        file - absolute file path
        Throws:
        java.lang.Exception

      • getDiagnosticsForFile

        public java.util.List<Diagnostic> getDiagnosticsForFile(java.lang.String file)
        Returns the list, possibly empty but never null, of errors and warnings generated in the file.

      • generateOutline

        public java.util.List<Outliner.Entry> generateOutline(java.lang.String path)
                                               throws java.lang.Exception
        Create an outline for a file in the index.
        Parameters:
        path - the file for which to build the outline
        Returns:
        a list of entries constituting the file outline. Returns an empty list if the indexer hasn't indexed that path.
        Throws:
        java.lang.Exception

      • putLocation

        public void putLocation(NNode node,
                        NBinding b)
        Add a reference to binding b at AST node node.
        Parameters:
        node - a node referring to a name binding. Typically a NName, NStr or NUrl.

      • putLocation

        public void putLocation(Ref ref,
                        NBinding b)

      • updateLocation

        public void updateLocation(Ref node,
                           NBinding b)
        Add node as a reference to binding b, removing node from any other binding ref-lists that it may have occupied. Currently only used in retargeting attribute references from provisional bindings once the actual binding is determined.

      • removeBinding

        public void removeBinding(NBinding b)

      • putProblem

        public void putProblem(NNode loc,
                       java.lang.String msg)

      • putProblem

        public void putProblem(java.lang.String file,
                       int beg,
                       int end,
                       java.lang.String msg)

      • loadFile

        public NModuleType loadFile(java.lang.String path)
                     throws java.lang.Exception
        Loads a file and all its ancestor packages.
        Throws:
        java.lang.Exception
        See Also:
        loadFile(String,boolean)

      • loadString

        public NModuleType loadString(java.lang.String path,
                              java.lang.String contents)
                       throws java.lang.Exception
        Loads a module from a string containing the module contents. Idempotent: looks in the module cache first. Used for simple unit tests.
        Parameters:
        path - a path for reporting/caching purposes. The filename component is used to construct the module qualified name.
        Throws:
        java.lang.Exception

      • loadFile

        public NModuleType loadFile(java.lang.String path,
                            boolean skipChain)
                     throws java.lang.Exception
        Load, parse and analyze a source file given its absolute path. By default, loads the entire ancestor package chain.
        Parameters:
        path - the absolute path to the file or directory. If it is a directory, it is suffixed with "__init__.py", and only that file is loaded from the directory.
        skipChain - true to skip loading ancestor packages
        Returns:
        null if path could not be loaded
        Throws:
        java.lang.Exception

      • getAstForFile

        public NModule getAstForFile(java.lang.String file)
                      throws java.lang.Exception
        Returns the syntax tree for file.

        Throws:
        java.lang.Exception

      • getAstForFile

        public NModule getAstForFile(java.lang.String file,
                             java.lang.String contents)
                      throws java.lang.Exception
        Returns the syntax tree for file.

        Throws:
        java.lang.Exception

      • getBuiltinModule

        public NModuleType getBuiltinModule(java.lang.String qname)
                             throws java.lang.Exception
        Throws:
        java.lang.Exception

      • loadModule

        public NModuleType loadModule(java.lang.String modname)
                       throws java.lang.Exception
        This method searches the module path for the module modname. If found, it is passed to loadFile(java.lang.String).

        The mechanisms for importing modules are in general statically undecidable. We make a reasonable effort to follow the most common lookup rules.

        Parameters:
        modname - a module name. Can be a relative path to a directory or a file (without the extension) or a possibly-qualified module name. If it is a module name, cannot contain leading dots.
        Throws:
        java.lang.Exception
        See Also:
        The Import Statement

      • loadFileRecursive

        public void loadFileRecursive(java.lang.String fullname)
                       throws java.lang.Exception
        Load all Python source files recursively if the given fullname is a directory; otherwise just load a file. Looks at file extension to determine whether to load a given file.
        Throws:
        java.lang.Exception

      • ready

        public void ready()
        Performs final indexing-building passes, including marking references to undeclared variables. Caller should invoke this method after loading all files.

      • clearAstCache

        public void clearAstCache()
        Clears the AST cache (to free up memory). Subsequent calls to getAstForFile(java.lang.String) will either fetch the serialized AST from a disk cache or re-parse the file from scratch.

      • clearModuleTable

        public void clearModuleTable()
        Clears the module table, discarding all resolved ASTs (modules) and their scope information.

      • recordUnresolvedModule

        public void recordUnresolvedModule(java.lang.String qname,
                                   java.lang.String file)
        Reports a failed module or submodule resolution.
        Parameters:
        qname - module qname, e.g. "org.foo.bar"
        file - the file where the unresolved import occurred

      • getStatusReport

        public java.lang.String getStatusReport()
        Report resolution rate and other statistics data.

      • numFilesLoaded

        public int numFilesLoaded()

      • getLoadedFiles

        public java.util.List<java.lang.String> getLoadedFiles()

      • log

        public void log(java.util.logging.Level level,
                java.lang.String msg)

      • severe

        public void severe(java.lang.String msg)

      • warn

        public void warn(java.lang.String msg)

      • info

        public void info(java.lang.String msg)

      • fine

        public void fine(java.lang.String msg)

      • finer

        public void finer(java.lang.String msg)

      • release

        public void release()
        Releases all resources for the current indexer.

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object
Skip navigation links

DMelt 3.0 © DataMelt by jWork.ORG

You see the box below because you did not login.