Grammar

Description

Grammar that tokenizes lines of text.

This class should not be instantiated directly but instead obtained from a GrammarRegistry by calling GrammarRegistry::loadGrammar.

API documentation

All methods

::onDidUpdate(callback)

Invoke the given callback when this grammar is updated due to a grammar it depends on being added or removed from the registry.

Argument Description
callback

Function to call when this grammar is updated.
Return values

  • Returns a Disposable on which .dispose() can be called to unsubscribe.

::tokenizeLines(text)

Tokenize all lines in the given text.

Argument Description
text

A String containing one or more lines.
Return values

  • Returns an Array of token arrays for each line tokenizied.

::tokenizeLine(line, ruleStack, firstLine)

Tokenize the line of text.

Argument Description
line

A String of text to tokenize.
ruleStack

An optional Array of rules previously returned from this method. This should be null when tokenizing the first line in the file.
firstLine

A optional Boolean denoting whether this is the first line in the file which defaults to false. This should be true when tokenizing the first line in the file.
Return values

  • Returns an Object containing the following properties:

    • line The String of text that was tokenized.
    • tags An Array of integer scope ids and strings. Positive ids indicate the beginning of a scope, and negative tags indicate the end. To resolve ids to scope names, call GrammarRegistry::scopeForId with the absolute value of the id.
    • tokens This is a dynamic property. Invoking it will incur additional overhead, but will automatically translate the tags into token objects with value and scopes properties.
    • ruleStack An Array of rules representing the tokenized state at the end of the line. These should be passed back into this method when tokenizing the next line in the file.