1324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver/* 2324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver [The 'BSD licence'] 3324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver Copyright (c) 2004 Terence Parr and Loring Craymer 4324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver All rights reserved. 5324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 6324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver Redistribution and use in source and binary forms, with or without 7324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver modification, are permitted provided that the following conditions 8324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver are met: 9324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 1. Redistributions of source code must retain the above copyright 10324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver notice, this list of conditions and the following disclaimer. 11324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 2. Redistributions in binary form must reproduce the above copyright 12324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver notice, this list of conditions and the following disclaimer in the 13324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver documentation and/or other materials provided with the distribution. 14324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 3. The name of the author may not be used to endorse or promote products 15324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver derived from this software without specific prior written permission. 16324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 17324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR 18324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 19324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 20324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, 21324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 22324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 23324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 24324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 25324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 26324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver*/ 28324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 29324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver/** Python 2.3.3 Grammar 30324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * 31324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * Terence Parr and Loring Craymer 32324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * February 2004 33324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * 34324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * Converted to ANTLR v3 November 2005 by Terence Parr. 35324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * 36324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * This grammar was derived automatically from the Python 2.3.3 37324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * parser grammar to get a syntactically correct ANTLR grammar 38324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * for Python. Then Terence hand tweaked it to be semantically 39324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * correct; i.e., removed lookahead issues etc... It is LL(1) 40324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * except for the (sometimes optional) trailing commas and semi-colons. 41324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * It needs two symbols of lookahead in this case. 42324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * 43324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * Starting with Loring's preliminary lexer for Python, I modified it 44324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * to do my version of the whole nasty INDENT/DEDENT issue just so I 45324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * could understand the problem better. This grammar requires 46324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * PythonTokenStream.java to work. Also I used some rules from the 47324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * semi-formal grammar on the web for Python (automatically 48324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * translated to ANTLR format by an ANTLR grammar, naturally <grin>). 49324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * The lexical rules for python are particularly nasty and it took me 50324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * a long time to get it 'right'; i.e., think about it in the proper 51324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * way. Resist changing the lexer unless you've used ANTLR a lot. ;) 52324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * 53324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * I (Terence) tested this by running it on the jython-2.1/Lib 54324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * directory of 40k lines of Python. 55324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * 56324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * REQUIRES ANTLR v3 57324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver */ 58324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvergrammar Python; 59324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruveroptions {language=JavaScript;} 60324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 61324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvertokens { 62324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver INDENT; 63324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver DEDENT; 64324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver} 65324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 66324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver@lexer::members { 67324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver/** Handles context-sensitive lexing of implicit line joining such as 68324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * the case where newline is ignored in cases like this: 69324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * a = [3, 70324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * 4] 71324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver */ 72324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver this.implicitLineJoiningLevel= 0; 73324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver this.startPos = -1; 74324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver} 75324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 76324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruversingle_input 77324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : NEWLINE 78324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | simple_stmt 79324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | compound_stmt NEWLINE 80324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 81324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 82324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverfile_input 83324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : (NEWLINE | stmt)* 84324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 85324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 86324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvereval_input 87324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : (NEWLINE)* testlist (NEWLINE)* 88324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 89324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 90324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverfuncdef 91324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : 'def' NAME parameters COLON suite 92324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver {xlog("found method def "+$NAME.text);} 93324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 94324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 95324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverparameters 96324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : LPAREN (varargslist)? RPAREN 97324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 98324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 99324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvervarargslist 100324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : defparameter (options {greedy=true;}:COMMA defparameter)* 101324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver (COMMA 102324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ( STAR NAME (COMMA DOUBLESTAR NAME)? 103324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | DOUBLESTAR NAME 104324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver )? 105324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver )? 106324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | STAR NAME (COMMA DOUBLESTAR NAME)? 107324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | DOUBLESTAR NAME 108324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 109324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 110324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverdefparameter 111324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : fpdef (ASSIGN test)? 112324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 113324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 114324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverfpdef 115324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : NAME 116324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | LPAREN fplist RPAREN 117324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 118324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 119324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverfplist 120324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : fpdef (options {greedy=true;}:COMMA fpdef)* (COMMA)? 121324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 122324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 123324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 124324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverstmt: simple_stmt 125324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | compound_stmt 126324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 127324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 128324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruversimple_stmt 129324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : small_stmt (options {greedy=true;}:SEMI small_stmt)* (SEMI)? NEWLINE 130324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 131324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 132324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruversmall_stmt: expr_stmt 133324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | print_stmt 134324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | del_stmt 135324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | pass_stmt 136324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | flow_stmt 137324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | import_stmt 138324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | global_stmt 139324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | exec_stmt 140324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | assert_stmt 141324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 142324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 143324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverexpr_stmt 144324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : testlist 145324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ( augassign testlist 146324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | (ASSIGN testlist)+ 147324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver )? 148324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 149324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 150324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruveraugassign 151324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : PLUSEQUAL 152324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | MINUSEQUAL 153324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | STAREQUAL 154324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | SLASHEQUAL 155324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | PERCENTEQUAL 156324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | AMPEREQUAL 157324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | VBAREQUAL 158324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | CIRCUMFLEXEQUAL 159324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | LEFTSHIFTEQUAL 160324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | RIGHTSHIFTEQUAL 161324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | DOUBLESTAREQUAL 162324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | DOUBLESLASHEQUAL 163324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 164324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 165324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverprint_stmt: 166324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 'print' 167324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ( testlist 168324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | RIGHTSHIFT testlist 169324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver )? 170324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 171324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 172324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverdel_stmt: 'del' exprlist 173324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 174324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 175324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverpass_stmt: 'pass' 176324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 177324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 178324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverflow_stmt: break_stmt 179324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | continue_stmt 180324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | return_stmt 181324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | raise_stmt 182324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | yield_stmt 183324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 184324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 185324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverbreak_stmt: 'break' 186324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 187324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 188324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvercontinue_stmt: 'continue' 189324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 190324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 191324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverreturn_stmt: 'return' (testlist)? 192324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 193324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 194324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruveryield_stmt: 'yield' testlist 195324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 196324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 197324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverraise_stmt: 'raise' (test (COMMA test (COMMA test)?)?)? 198324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 199324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 200324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverimport_stmt 201324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : 'import' dotted_as_name (COMMA dotted_as_name)* 202324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | 'from' dotted_name 'import' 203324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver (STAR | import_as_name (COMMA import_as_name)*) 204324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 205324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 206324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverimport_as_name 207324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : NAME (NAME NAME)? 208324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 209324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 210324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverdotted_as_name: dotted_name (NAME NAME)? 211324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 212324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 213324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverdotted_name: NAME (DOT NAME)* 214324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 215324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 216324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverglobal_stmt: 'global' NAME (COMMA NAME)* 217324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 218324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 219324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverexec_stmt: 'exec' expr ('in' test (COMMA test)?)? 220324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 221324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 222324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverassert_stmt: 'assert' test (COMMA test)? 223324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 224324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 225324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 226324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvercompound_stmt: if_stmt 227324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | while_stmt 228324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | for_stmt 229324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | try_stmt 230324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | funcdef 231324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | classdef 232324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 233324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 234324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverif_stmt: 'if' test COLON suite ('elif' test COLON suite)* ('else' COLON suite)? 235324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 236324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 237324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverwhile_stmt: 'while' test COLON suite ('else' COLON suite)? 238324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 239324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 240324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverfor_stmt: 'for' exprlist 'in' testlist COLON suite ('else' COLON suite)? 241324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 242324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 243324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvertry_stmt 244324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : 'try' COLON suite 245324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ( (except_clause COLON suite)+ ('else' COLON suite)? 246324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | 'finally' COLON suite 247324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ) 248324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 249324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 250324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverexcept_clause: 'except' (test (COMMA test)?)? 251324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 252324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 253324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruversuite: simple_stmt 254324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | NEWLINE INDENT (stmt)+ DEDENT 255324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 256324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 257324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 258324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvertest: and_test ('or' and_test)* 259324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | lambdef 260324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 261324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 262324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverand_test 263324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : not_test ('and' not_test)* 264324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 265324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 266324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvernot_test 267324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : 'not' not_test 268324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | comparison 269324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 270324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 271324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvercomparison: expr (comp_op expr)* 272324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 273324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 274324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvercomp_op: LESS 275324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver |GREATER 276324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver |EQUAL 277324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver |GREATEREQUAL 278324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver |LESSEQUAL 279324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver |ALT_NOTEQUAL 280324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver |NOTEQUAL 281324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver |'in' 282324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver |'not' 'in' 283324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver |'is' 284324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver |'is' 'not' 285324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 286324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 287324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverexpr: xor_expr (VBAR xor_expr)* 288324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 289324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 290324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverxor_expr: and_expr (CIRCUMFLEX and_expr)* 291324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 292324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 293324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverand_expr: shift_expr (AMPER shift_expr)* 294324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 295324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 296324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvershift_expr: arith_expr ((LEFTSHIFT|RIGHTSHIFT) arith_expr)* 297324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 298324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 299324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverarith_expr: term ((PLUS|MINUS) term)* 300324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 301324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 302324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverterm: factor ((STAR | SLASH | PERCENT | DOUBLESLASH ) factor)* 303324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 304324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 305324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverfactor 306324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : (PLUS|MINUS|TILDE) factor 307324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | power 308324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 309324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 310324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverpower 311324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : atom (trailer)* (options {greedy=true;}:DOUBLESTAR factor)? 312324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 313324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 314324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruveratom: LPAREN (testlist)? RPAREN 315324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | LBRACK (listmaker)? RBRACK 316324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | LCURLY (dictmaker)? RCURLY 317324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | BACKQUOTE testlist BACKQUOTE 318324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | NAME 319324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | INT 320324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | LONGINT 321324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | FLOAT 322324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | COMPLEX 323324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | (STRING)+ 324324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 325324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 326324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverlistmaker: test ( list_for | (options {greedy=true;}:COMMA test)* ) (COMMA)? 327324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 328324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 329324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverlambdef: 'lambda' (varargslist)? COLON test 330324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 331324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 332324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvertrailer: LPAREN (arglist)? RPAREN 333324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | LBRACK subscriptlist RBRACK 334324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | DOT NAME 335324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 336324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 337324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruversubscriptlist 338324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : subscript (options {greedy=true;}:COMMA subscript)* (COMMA)? 339324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 340324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 341324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruversubscript 342324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : DOT DOT DOT 343324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | test (COLON (test)? (sliceop)?)? 344324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | COLON (test)? (sliceop)? 345324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 346324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 347324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruversliceop: COLON (test)? 348324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 349324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 350324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverexprlist 351324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : expr (options {k=2;}:COMMA expr)* (COMMA)? 352324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 353324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 354324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruvertestlist 355324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : test (options {k=2;}: COMMA test)* (COMMA)? 356324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 357324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 358324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverdictmaker 359324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : test COLON test 360324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver (options {k=2;}:COMMA test COLON test)* (COMMA)? 361324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 362324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 363324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverclassdef: 'class' NAME (LPAREN testlist RPAREN)? COLON suite 364324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver {xlog("found class def "+$NAME.text);} 365324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 366324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 367324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverarglist: argument (COMMA argument)* 368324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ( COMMA 369324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ( STAR test (COMMA DOUBLESTAR test)? 370324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | DOUBLESTAR test 371324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver )? 372324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver )? 373324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | STAR test (COMMA DOUBLESTAR test)? 374324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | DOUBLESTAR test 375324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 376324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 377324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverargument : test (ASSIGN test)? 378324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 379324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 380324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverlist_iter: list_for 381324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | list_if 382324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 383324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 384324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverlist_for: 'for' exprlist 'in' testlist (list_iter)? 385324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 386324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 387324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverlist_if: 'if' test (list_iter)? 388324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 389324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 390324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverLPAREN : '(' {this.implicitLineJoiningLevel++;} ; 391324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 392324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverRPAREN : ')' {this.implicitLineJoiningLevel--;} ; 393324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 394324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverLBRACK : '[' {this.implicitLineJoiningLevel++;} ; 395324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 396324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverRBRACK : ']' {this.implicitLineJoiningLevel--;} ; 397324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 398324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverCOLON : ':' ; 399324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 400324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverCOMMA : ',' ; 401324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 402324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverSEMI : ';' ; 403324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 404324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverPLUS : '+' ; 405324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 406324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverMINUS : '-' ; 407324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 408324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverSTAR : '*' ; 409324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 410324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverSLASH : '/' ; 411324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 412324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverVBAR : '|' ; 413324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 414324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverAMPER : '&' ; 415324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 416324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverLESS : '<' ; 417324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 418324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverGREATER : '>' ; 419324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 420324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverASSIGN : '=' ; 421324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 422324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverPERCENT : '%' ; 423324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 424324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverBACKQUOTE : '`' ; 425324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 426324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverLCURLY : '{' {this.implicitLineJoiningLevel++;} ; 427324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 428324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverRCURLY : '}' {this.implicitLineJoiningLevel--;} ; 429324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 430324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverCIRCUMFLEX : '^' ; 431324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 432324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverTILDE : '~' ; 433324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 434324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverEQUAL : '==' ; 435324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 436324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverNOTEQUAL : '!=' ; 437324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 438324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverALT_NOTEQUAL: '<>' ; 439324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 440324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverLESSEQUAL : '<=' ; 441324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 442324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverLEFTSHIFT : '<<' ; 443324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 444324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverGREATEREQUAL : '>=' ; 445324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 446324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverRIGHTSHIFT : '>>' ; 447324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 448324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverPLUSEQUAL : '+=' ; 449324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 450324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverMINUSEQUAL : '-=' ; 451324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 452324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverDOUBLESTAR : '**' ; 453324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 454324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverSTAREQUAL : '*=' ; 455324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 456324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverDOUBLESLASH : '//' ; 457324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 458324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverSLASHEQUAL : '/=' ; 459324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 460324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverVBAREQUAL : '|=' ; 461324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 462324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverPERCENTEQUAL : '%=' ; 463324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 464324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverAMPEREQUAL : '&=' ; 465324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 466324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverCIRCUMFLEXEQUAL : '^=' ; 467324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 468324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverLEFTSHIFTEQUAL : '<<=' ; 469324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 470324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverRIGHTSHIFTEQUAL : '>>=' ; 471324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 472324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverDOUBLESTAREQUAL : '**=' ; 473324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 474324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverDOUBLESLASHEQUAL : '//=' ; 475324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 476324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverDOT : '.' ; 477324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 478324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverFLOAT 479324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : '.' DIGITS (Exponent)? 480324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | DIGITS ('.' (DIGITS (Exponent)?)? | Exponent) 481324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 482324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 483324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverLONGINT 484324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : INT ('l'|'L') 485324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 486324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 487324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverfragment 488324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverExponent 489324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : ('e' | 'E') ( '+' | '-' )? DIGITS 490324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 491324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 492324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverINT : // Hex 493324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver '0' ('x' | 'X') ( '0' .. '9' | 'a' .. 'f' | 'A' .. 'F' )+ 494324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ('l' | 'L')? 495324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | // Octal 496324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver '0' DIGITS* 497324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | '1'..'9' DIGITS* 498324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 499324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 500324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverCOMPLEX 501324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : INT ('j'|'J') 502324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | FLOAT ('j'|'J') 503324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 504324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 505324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverfragment 506324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverDIGITS : ( '0' .. '9' )+ ; 507324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 508324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverNAME: ( 'a' .. 'z' | 'A' .. 'Z' | '_') 509324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ( 'a' .. 'z' | 'A' .. 'Z' | '_' | '0' .. '9' )* 510324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 511324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 512324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver/** Match various string types. Note that greedy=false implies ''' 513324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * should make us exit loop not continue. 514324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver */ 515324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverSTRING 516324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : ('r'|'u'|'ur')? 517324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ( '\'\'\'' (options {greedy=false;}:.)* '\'\'\'' 518324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | '"""' (options {greedy=false;}:.)* '"""' 519324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | '"' (ESC|~('\\'|'\n'|'"'))* '"' 520324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | '\'' (ESC|~('\\'|'\n'|'\''))* '\'' 521324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ) 522324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 523324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 524324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruverfragment 525324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverESC 526324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : '\\' . 527324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 528324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 529324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver/** Consume a newline and any whitespace at start of next line */ 530324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverCONTINUED_LINE 531324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : '\\' ('\r')? '\n' (' '|'\t')* { $channel=HIDDEN; } 532324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 533324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 534324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver/** Treat a sequence of blank lines as a single blank line. If 535324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * nested within a (..), {..}, or [..], then ignore newlines. 536324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * If the first newline starts in column one, they are to be ignored. 537324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver */ 538324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverNEWLINE 539324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : (('\r')? '\n' )+ 540324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver {if ( this.startPos==0 || this.implicitLineJoiningLevel>0 ) 541324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver $channel=HIDDEN; 542324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver } 543324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 544324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 545324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverWS : {this.startPos>0}?=> (' '|'\t')+ {$channel=HIDDEN;} 546324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 547324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 548324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver/** Grab everything before a real symbol. Then if newline, kill it 549324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * as this is a blank line. If whitespace followed by comment, kill it 550324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * as it's a comment on a line by itself. 551324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * 552324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver * Ignore leading whitespace when nested in [..], (..), {..}. 553324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver */ 554324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverLEADING_WS 555324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver@init { 556324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver var spaces = 0; 557324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver} 558324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : {this.startPos==0}?=> 559324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ( {this.implicitLineJoiningLevel>0}? ( ' ' | '\t' )+ {$channel=HIDDEN;} 560324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | ( ' ' { spaces++; } 561324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | '\t' { spaces += 8; spaces -= (spaces \% 8); } 562324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver )+ 563324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver { 564324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver // make a string of n spaces where n is column number - 1 565324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver var indentation = new Array(spaces); 566324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver for (var i=0; i<spaces; i++) { 567324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver indentation[i] = ' '; 568324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver } 569324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver var s = indentation.join(""); 570324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver this.emit(new org.antlr.runtime.CommonToken(this.LEADING_WS,s)); 571324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver } 572324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver // kill trailing newline if present and then ignore 573324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ( ('\r')? '\n' {if (this.state.token!=null) this.state.token.setChannel(HIDDEN); else $channel=HIDDEN;})* 574324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver // {this.token.setChannel(99); } 575324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ) 576324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 577324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 578324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver/** Comments not on line by themselves are turned into newlines. 579324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 580324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver b = a # end of line comment 581324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 582324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver or 583324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 584324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver a = [1, # weird 585324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 2] 586324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 587324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver This rule is invoked directly by nextToken when the comment is in 588324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver first column or when comment is on end of nonwhitespace line. 589324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 590324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver Only match \n here if we didn't start on left edge; let NEWLINE return that. 591324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver Kill if newlines if we live on a line by ourselves 592324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver 593324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver Consume any leading whitespace if it starts on left edge. 594324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver */ 595324c4644fee44b9898524c09511bd33c3f12e2dfBen GruverCOMMENT 596324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver@init { 597324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver $channel=HIDDEN; 598324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver} 599324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver : {this.startPos==0}?=> (' '|'\t')* '#' (~'\n')* '\n'+ 600324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver | {this.startPos>0}?=> '#' (~'\n')* // let NEWLINE handle \n unless char pos==0 for '#' 601324c4644fee44b9898524c09511bd33c3f12e2dfBen Gruver ; 602