Introduction to ANTLR4
Hannes Kinks
hkinks@gmail.com
Tallinn 2016
Resources
- Homepage - http://www.antlr.org
- Book - https://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference
- Presentation about ANTLR4 by Terence Parr -
http://www.youtube.com/watch?v=q8p1voEiu8Q
GIT
picoCPU compiler project
https://bitbucket.org/hkinks/iag0450-antlr
Tutorial project
https://bitbucket.org/hkinks/antlr-tutorial
Compiler made in 2013
https://bitbucket.org/hkinks/estcompiler
ANTLR4 vs Flex+Bison
- Ease of use over performance
- GUI tools for visualization
- Automatic generation of parse trees
- Decoupled grammar and application-specific code
- Listeners
- Visitors
- Different parsing algorithms
- LL(*) vs LALR
Parsing algorithm comparison
Bison - LALR
Bottom-up parsing
ANTLR - LL(*)
Top-down parsing
LALR - Look-Ahead Left to Right
LL(k) - Left to right, performing Leftmost derivation, with k tokens of lookahead
LL(k) parser
https://en.wikipedia.org/wiki/LL_parser
Given context-free grammar
- S -> E
- E -> (E+E)
- E -> i
((i+i)+i)
Text for parsing
S=>E
=>(E+E)
=>((E+E)+E)
=>((i+E)+E)
=>((i+i)+E)
=>((i+i)+i)
Derivation process:
k element lookahead
E=>(E+E)
or
E=>i
?
Answer: look next symbol
Parsing with ANTLR4
- Grammar has rules both for:
- Lexer
- Parser
- Described in *.g4 file
1. Define grammar in Assign.g4
Parsing with ANTLR
2. ANTLR tool generates code based on specified grammar
- Lexer
- Parser
- Tokens
Parsing with ANTLR
3. Parsing given input
Parsing with ANTLR
4. Walking the parse tree
- Listener pattern
- Visitor pattern
Listener
- Parse tree walked depth-first
- Events fired at every node
- Entering node
- Exiting node
- Visiting terminal node
Visitor pattern
- Option -visitor creates visitor
- Having control over the 'walk'
- Nodes are visited explicitly by calling out their methods
ParseTree tree = ... ; // tree is result of parsing
MyVisitor v = new MyVisitor();
v.visit(tree);
Setting up the environment
- Step-by-step tutorial
- Files: http://ati.ttu.ee/~hkinks/antlr/
-
antlr.sh
- run with 'sh antlr.sh'
- downloads ANTLR4 library
- adds lib to classpath
- sets aliases for antlr4 and grun
-
plugin.zip
- Plugin for Intellij IDEA
- Plugins existing also for eclipse and netbeans
Individual task
JSON->XML parser
- Define grammar in *.g4
- Generate lexer and parser
- Walk the tree with Listener pattern
- Write XML into file