• Home
• com.tunnelvisionlabs
• antlr4-perf-testsuite
• 4.7.1
• source code
• TokenSource.html

×

Project download

Many resources are needed to download a project. Please understand that we have to compensate our server costs. Thank you in advance.
Project price only 1 $

You can buy this project and download/modify it how often you want.

Java.target.apidocs.org.antlr.v4.runtime.TokenSource.html Maven / Gradle / Ivy

Overview
Package
Class
Use
Tree
Deprecated
Index
Help




Prev Class
Next Class


Frames
No Frames


All Classes






Summary: 
Nested | 
Field | 
Constr | 
Method


Detail: 
Field | 
Constr | 
Method








org.antlr.v4.runtime
Interface TokenSource






All Known Implementing Classes:
Lexer, LexerInterpreter, ListTokenSource, XPathLexer




public interface TokenSource
A source of tokens must provide a sequence of tokens via nextToken()
 and also must reveal it's source of characters; CommonToken's text is
 computed from a CharStream; it only store indices into the char
 stream.

 
Errors from the lexer are never passed to the parser. Either you want to keep
 going or you do not upon token recognition error. If you do not want to
 continue lexing then you do not want to continue parsing. Just throw an
 exception not under RecognitionException and Java will naturally toss
 you all the way out of the recognizers. If you want to continue lexing then
 you should not throw an exception to the parser--it has already requested a
 token. Keep lexing until you get a valid one. Just report errors and keep
 going, looking for a valid token.











Method Summary

Methods 

Modifier and Type
Method and Description


int
getCharPositionInLine()
Get the index into the current line for the current position in the input
 stream.



CharStream
getInputStream()
Get the CharStream from which this token source is currently
 providing tokens.



int
getLine()
Get the line number for the current position in the input stream.



String
getSourceName()
Gets the name of the underlying input source.



TokenFactory
getTokenFactory()
Gets the TokenFactory this token source is currently using for
 creating Token objects from the input.



Token
nextToken()
Return a Token object from your input stream (usually a
 CharStream).



void
setTokenFactory(TokenFactory factory)
Set the TokenFactory this token source should use for creating
 Token objects from the input.
















Method Detail





nextToken
@NotNull
Token nextToken()
Return a Token object from your input stream (usually a
 CharStream). Do not fail/return upon lexing error; keep chewing
 on the characters until you get a good one; errors are not passed through
 to the parser.







getLine
int getLine()
Get the line number for the current position in the input stream. The
 first line in the input is line 1.
Returns:
The line number for the current position in the input stream, or
 0 if the current token source does not track line numbers.







getCharPositionInLine
int getCharPositionInLine()
Get the index into the current line for the current position in the input
 stream. The first character on a line has position 0.
Returns:
The line number for the current position in the input stream, or
 -1 if the current token source does not track character positions.







getInputStream
@Nullable
CharStream getInputStream()
Get the CharStream from which this token source is currently
 providing tokens.
Returns:
The CharStream associated with the current position in
 the input, or null if no input stream is available for the token
 source.







getSourceName
@NotNull
String getSourceName()
Gets the name of the underlying input source. This method returns a
 non-null, non-empty string. If such a name is not known, this method
 returns IntStream.UNKNOWN_SOURCE_NAME.







setTokenFactory
void setTokenFactory(@NotNull
                   TokenFactory factory)
Set the TokenFactory this token source should use for creating
 Token objects from the input.
Parameters:
factory - The TokenFactory to use for creating tokens.







getTokenFactory
@NotNull
TokenFactory getTokenFactory()
Gets the TokenFactory this token source is currently using for
 creating Token objects from the input.
Returns:
The TokenFactory currently used by this token source.
















Overview
Package
Class
Use
Tree
Deprecated
Index
Help




Prev Class
Next Class


Frames
No Frames


All Classes






Summary: 
Nested | 
Field | 
Constr | 
Method


Detail: 
Field | 
Constr | 
Method






Copyright © 1992–2018 Tunnel Vision Laboratories, LLC. All rights reserved.