Java.target.apidocs.org.antlr.v4.runtime.CommonTokenStream.html Maven / Gradle / Ivy
Go to download
Show more of this group Show more artifacts with this name
Show all versions of antlr4-perf-testsuite Show documentation
Show all versions of antlr4-perf-testsuite Show documentation
The ANTLR 4 grammar compiler.
CommonTokenStream (ANTLR 4 Runtime (Optimized) 4.13.1.3 API)
org.antlr.v4.runtime
Class CommonTokenStream
- java.lang.Object
-
- org.antlr.v4.runtime.BufferedTokenStream
-
- org.antlr.v4.runtime.CommonTokenStream
-
- All Implemented Interfaces:
- IntStream, TokenStream
public class CommonTokenStream
extends BufferedTokenStream
This class extends BufferedTokenStream with functionality to filter
token streams to tokens on a particular channel (tokens where
Token.getChannel() returns a particular value).
This token stream provides access to all tokens by index or when calling
methods like BufferedTokenStream.getText(). The channel filtering is only used for code
accessing tokens via the lookahead methods BufferedTokenStream.LA(int), LT(int), and
LB(int).
By default, tokens are placed on the default channel
(Token.DEFAULT_CHANNEL), but may be reassigned by using the
->channel(HIDDEN) lexer command, or by using an embedded action to
call Lexer.setChannel(int).
Note: lexer rules which use the ->skip lexer command or call
Lexer.skip() do not produce tokens at all, so input text matched by
such a rule will not be available as part of the token stream, regardless of
channel.
-
-
Field Summary
Fields
Modifier and Type
Field and Description
protected intchannel
Specifies the channel to use for filtering tokens.
-
Fields inherited from class org.antlr.v4.runtime.BufferedTokenStream
fetchedEOF, p, tokens, tokenSource
-
Fields inherited from interface org.antlr.v4.runtime.IntStream
EOF, UNKNOWN_SOURCE_NAME
-
Constructor Summary
Constructors
Constructor and Description
CommonTokenStream(TokenSource tokenSource)
Constructs a new CommonTokenStream using the specified token
source and the default token channel (Token.DEFAULT_CHANNEL).
CommonTokenStream(TokenSource tokenSource,
int channel)
Constructs a new CommonTokenStream using the specified token
source and filtering tokens to the specified channel.
-
Method Summary
All Methods Instance Methods Concrete Methods
Modifier and Type
Method and Description
protected intadjustSeekIndex(int i)
Allowed derived classes to modify the behavior of operations which change
the current stream position by adjusting the target token index of a seek
operation.
intgetNumberOfOnChannelTokens()
Count EOF just once.
protected TokenLB(int k)
TokenLT(int k)
-
Methods inherited from class org.antlr.v4.runtime.BufferedTokenStream
consume, fetch, fill, filterForChannel, get, get, getHiddenTokensToLeft, getHiddenTokensToLeft, getHiddenTokensToRight, getHiddenTokensToRight, getSourceName, getText, getText, getText, getText, getTokens, getTokens, getTokens, getTokens, getTokenSource, index, LA, lazyInit, mark, nextTokenOnChannel, previousTokenOnChannel, release, reset, seek, setTokenSource, setup, size, sync
-
-
Field Detail
-
channel
protected int channel
Specifies the channel to use for filtering tokens.
The default value is Token.DEFAULT_CHANNEL, which matches the
default channel assigned to tokens created by the lexer.
-
Constructor Detail
-
CommonTokenStream
public CommonTokenStream(@NotNull
TokenSource tokenSource)
Constructs a new CommonTokenStream using the specified token
source and the default token channel (Token.DEFAULT_CHANNEL).
- Parameters:
tokenSource - The token source.
-
CommonTokenStream
public CommonTokenStream(@NotNull
TokenSource tokenSource,
int channel)
Constructs a new CommonTokenStream using the specified token
source and filtering tokens to the specified channel. Only tokens whose
Token.getChannel() matches channel or have the
Token.getType() equal to Token.EOF will be returned by the
token stream lookahead methods.
- Parameters:
tokenSource - The token source.channel - The channel to use for filtering tokens.
-
Method Detail
-
adjustSeekIndex
protected int adjustSeekIndex(int i)
Description copied from class: BufferedTokenStream
Allowed derived classes to modify the behavior of operations which change
the current stream position by adjusting the target token index of a seek
operation. The default implementation simply returns i. If an
exception is thrown in this method, the current stream index should not be
changed.
For example, CommonTokenStream overrides this method to ensure that
the seek target is always an on-channel token.
- Overrides:
adjustSeekIndex in class BufferedTokenStream- Parameters:
i - The target token index.- Returns:
- The adjusted target token index.
-
LB
protected Token LB(int k)
- Overrides:
LB in class BufferedTokenStream
-
LT
public Token LT(int k)
Description copied from interface: TokenStream
Get the Token instance associated with the value returned by
LA(k). This method has the same pre- and post-conditions as
IntStream.LA(int). In addition, when the preconditions of this method
are met, the return value is non-null and the value of
LT(k).getType()==LA(k).
- Specified by:
LT in interface TokenStream- Overrides:
LT in class BufferedTokenStream- See Also:
IntStream.LA(int)
-
getNumberOfOnChannelTokens
public int getNumberOfOnChannelTokens()
Count EOF just once.
Copyright © 1992–2023 Daniel Sun. All rights reserved.