ANTLR Basic Tutorial

latest update: 2013-08-14


What is ANTLR

ANTLR == ANother Tool for Language Recognition

  • ANother
    • existing (comparable) ones: JavaCC
    • older ones: lex/yacc
  • Tool
    • a language tool
      • provides a framework for constructing
        • recognizers
        • interpreters
        • compilers
        • translators
  • Language Recognition
    • You write a specific syntax/grammar, then use ANTLR to recognize it
    • specific syntax/grammar: normally called domain (specific) language
      • Examples
        • XML
        • MySQL

ANTLR Features

  1. can gererate:
    • lexer
    • parser
    • tree parser (aka: tree walker)
  2. support error recovery and error reporting
  3. support multiple target language:Ada,ActionScript,C,C++,C#,D,Emacs ELisp,Objective C,Java,JavaScript ,Python,Ruby,Perl6,Perl,PHP,Oberon,Scala

ANTLR Version History and Basic Syntax

according to ANTLR development history, till now(2013-08-14), totoal 4 main versions:

ANTLR v1 == PCCTS

originally called PCCTS (Purdue Compiler Construction Tool Set), that is ANTLR v1.00, released in 1989

ATNLR v2

from (released May 1997) ANTLR v2.0.0 to ATNLR v2.7.5 (released January 28, 2005)

for ANTLR v2, grammar divided to two part:

  • lexer rule
  • parser rule

ANTLR v2 Sample Code

lexer sample:

class L extends Lexer; 
A   :   B 'b' 
    ; 
protected  // only called from another lex rule 
B   :   'x' ('b' | ) 
    ;

parser sample:

[TODO: find and add parer sample code]

ATNLR v3

currently, most widely used is ANTLR v3.

Homepage: http://www.antlr3.org/

ANTLR v3 support combined grammar of lexer and parser

ANTLR v3 Sample Code

grammar SimpleCalc; 
add : NUMBER PLUS NUMBER; 
NUMBER  : ('0'..'9')+ ; 
PLUS    : '+';

Note:

when you use ANTLR IDE(antlrworks 1.x.jar) to create .g grammar file, then let you choose grammar type, it will show you:lexer or parser, or combined(lexer and parser)

show combined grammar

ATNLR v4

latest version, but currenlt not widely used one is ANTLR v4.

Homepage:http://www.antlr.org/

ANTLR Development Tool

simple speaking, these two kind of antlr development environment:

  1. based on command line
  2. use IDE

ANTLR IDE

ANTLRWorks 1.x

for ANTLR v3 (and previous version), can use ANTRLWorks 1.x

latest version is: antlrworks-1.5rc2.jar

(also can found other version in antlr v3 org download page)

it is just a jar file:

on Windows plaftorm, can double click it, to run

Screenshot:

antlrworks 1.5rc2 ide

ANTLRWorks 2.x

based on NetBeans IDE, merge the ANTLR development functions into it.

Screenshot:

 antlrworks2 based on netbeans

can download ANTLRWorks2 from:

http://tunnelvisionlabs.com/products/demo/antlrworks

ANTLR Syntax

here is summary for ANTLR v3 syntax:

ANTLR Symbols Syntax

 

Symbol

Description

Example

(…)

subrule

 

(…)*

0~∞ closure subrule

 

(…)+

1~∞ closure subrule

 

(…)?

0 or 1 closure subrule

 

{…}

semantic action

(‘ ‘|’\t’)+ {skip();}

[…]

rule arguments

 

{…}?

semantic predicate

 

(…)=>

syntactic predicate

 

|

alternative operator

‘+’|’-’

..

range operator

’0′.. ’9′

~

not operator

 

.

wildcard

 

=

assignment operator

 

:

label operator, rule start

 

;

rule end

INT : ’0′..’9′+ ;

<…>

element option

 

class

grammar class

 

extends

specifies grammar base class

 

returns

specifies return type of rule

 

options

options section

 

tokens

tokens section

 

header

header section

 

tokens

token definition section

 

 

Production Element Operators:

Symbol

Description

Example

^

AST root operator

multExpr: atom (‘*’^ atom)* ;

!

AST exclude operator

atom : INT

| ‘(‘! expr ‘)’!

;

How do ANTLR development

ANTLR v3 development process

1.double click antlrworks-1.5rc2.jar

pure antlrworks 1.5 jar ui

2.create a new .g (antlr v3) grammar file ExprSimple.g then save

antlrworks file new

 new antlr 3 grammar .g file

 expr simple demo for antlr v3 grammar

 newly created antlr v3 grammar file

change to .g grammar to this sample code:

​grammar ExprSimple;

options {
    output = AST;
    ASTLabelType = CommonTree; // type of $stat.tree ref etc...
}

INT        :    '0'..'9'+ ;
NEWLINE    :    '\r'? '\n' ;
WS        :    (' '|'\t')+ {skip();} ;

prog    :    stat+ ;

stat    :    expr NEWLINE    -> expr;
   
expr    :    multExpr (('+'^|'-'^) multExpr)*;

multExpr:    atom ('*'^ atom)* ;

atom    :    INT |    '('! expr ')'! ;

like this:

 change to sample code

3.compile and debug .g file

Run->Debug

 run debug for sample code

then will note will first to compile your code:

note will compiling your code

Note:

after compile, antlrworks will generate the output folder under same folder with .g source code:

 generated output folder

it will contain the necessary xxxLexer.java and xxxParser.java:

 lexer java and parser java files

then let you input debut content, here use simpile text:

input expression to test

Note:

for large test content, normall we will choose the File instead of Text.

then will preparing for debug:

 then will preparing for debug

then can do debug:

 click step forward to debug

then parse tree will show result:

parse tree will dynamically show result

continue debug, to the end, will show all parse tree:

finaly show all tree nodes

ANTLR v4 development process

after download that ANTLRWorks2 ide, decompress it

run antlrworks2\bin\antlrworks264.exe( if your os is x86, then run antlrworks2\bin\antlrworks2.exe)

basically same with antlr v3 development.

first create new .g4 file:

antlrworks2 file new file

choose ANTLR->ANTLR 4 Combined Grammar:

 antlr antlr 4 combined grammar next

then Next, then input filename choose save folder:

 expr expr g4 file

then we can see newly created file:

 new created expr antlr 4 file

then change the content to:

​grammar Expr;

INT        :    '0'..'9'+ ;
NEWLINE    :    '\r'? '\n' ;

prog:    (expr NEWLINE)* ;
expr:    expr ('*'|'/') expr
        |    expr ('+'|'-') expr
    |    INT
    |    '(' expr ')'
    ;

like this:

antlr v4 test code

then continue to test:

Run-> Run in TestRig…

run run in testrig

then will need you to input test content, note here is not support directly input test data, only support input file, so need firstly to goto create a test file, like:

created expr text txt file

content is:

​1*2 + 3*4

then choose this test file ExprText.txt, and start rule choose prog
expr test txt prog finish

then will popup a window, show the parse result:

 then will popup window show parse tree

that is all the basic development process for antlr v4.

 

Some ANTLR Materials

1.antlr github

https://github.com/antlr

can found source code for antlr3, antlr4, antlr grammar samples….

2.antlr v3 download  page

http://www.antlr3.org/download/

can download many useful antlr libs, sourcecode, antlrworks ide, …



发表评论

电子邮件地址不会被公开。 必填项已用*标注

无觅相关文章插件,快速提升流量