Feeds

• Site
• News

How do I start?

With jparsec, one constructs Parser object in terms of the production rule of the grammar. Once a Parser object is created, it can be used as in:

Depending on your need, this return value can be either the calculation result or an abstract syntax tree.

So how does one create Parser object? The following are the most important classes:

• Parser
  
  A parser encapsulates a piece of parsing logic and simple Parser objects can be combined to create more complex parser.
• Parsers
  
  Common parser implementation.
• Scanners
  
  A scanner is a parser that scans the source string and recognizes certain patterns.
• Terminals
  
  Provides tokenizers and lexers for common terminals such as identifier, integer, scientific number etc.
• OperatorTable
  
  Supports operator precedence grammar. Programmer declares operators in an OperatorTable and the framework will take care of constructing a full-blown expression parser.

What are the top 5 combinators that I need to familiarize myself with?

• or
  
  This is the logical alternative combinator. A production rule "A ::= B|C|D" can be represented as:
• next/sequence
  
  This is the sequential combinator. Production rule "A ::= BCD" can be represented as:
  With alternative and sequential combinator, one can start building sophisticated parsers.
• map/sequence
  
  When constructing a parser, we typically want to not only recognize a certain grammar, but also to build some object or perform some computation based on the recognized grammar. This family of map/sequence combinators can be used to perform such computation. For example, in order to use the parser result of B, C, D to create an object of A, one can implement the callback interface Map3, which accepts the parser result of B, C and D as input parameter and returns the A object as result.
  
  Implementing anonymous class for the Map interfaces could be verbose though. A convenience Mapper class is provided to simplify the syntax. It requires additional dependency on cglib.
• many/many1
  
  These combinators implement the "kleene star" and "kleene cross" logic in BNF. "A ::= B*" is represented as:
  or
  where the latter will additionally return a list of Foo object as the parser result.
• lazy
  
  Production rules can involve recursion. (for example, an expression with binary operators is represented recursively in production rule). The lazy combinator allows a parser to reference a parser that will be set later.

Lexical analysis vs. syntactical analysis

In a simple scenario, all work can be done in the scanning phase. For example:

However, when the complexity of the grammar rule scales up and there are whitespaces and comments to ignore from the grammar, one-pass parsing becomes awkward. A 2-pass approach can then be used. That is, a lexical analysis phase scans the source as a list of Tokens and then a second syntactical analysis phase parses the tokens.

The Terminals class provides common tokenizers that scans the source string and turns them into tokens. It also provides corresponding syntactic parsers that recognize these tokens in the syntactical analysis phase.

A syntactical parser takes a list of tokens as input, this list needs to come from the output of a lexer. The Parser.from() API can be used to chain a syntactical parser with a lexer.

What are the typical steps in creating a conventional 2-pass parser?