Lines Matching refs:token
77 /** Track the set of token types that can follow any rule invocation.
84 * matched a token. Prevents generation of more than one error message
91 * but no token is consumed during recovery...another error is found,
93 * one token/tree node is consumed for two errors.
97 /** In lieu of a return value, this indicates that a rule or token
98 * has failed to match. Reset to false upon valid token match.
114 * Tracks the stop token index for each rule. ruleMemo[ruleIndex] is
116 * get back the stop token for associated rule or MEMO_RULE_FAILED.
122 /** Pointer to an array of token names
125 * at parser compile time - grammar token names are thus restricted in character
136 /** The goal of all lexer rules/methods is to create a token object.
138 * create a single token. For example, NUM : INT | FLOAT ;
139 * In this case, you want the INT or FLOAT rule to set token and not
140 * have it reset to a NUM token in rule NUM.
142 pANTLR3_COMMON_TOKEN token;
144 /** The goal of all lexer rules being to create a token, then a lexer
145 * needs to build a token factory to create them.
151 * a token source interface implementation.
155 /** The channel number for the current token
159 /** The token type for the current token
164 * token resides.
168 /** The character position of the first character of the current token
173 /** What character index in the stream did the current token start at?
174 * Needed, for example, to get the text for current token. Set at
179 /** Text for the current token. This can be overridden by setting this
186 * token.
197 /// A stack of token/tree rewrite streams that are available for use