Java.target.apidocs.org.antlr.v4.runtime.CommonTokenStream.html Maven / Gradle / Ivy

Go to download

Show more of this group Show more artifacts with this name
Show all versions of antlr4-perf-testsuite Show documentation

The ANTLR 4 grammar compiler.

There is a newer version: 4.9.0

Show newest version







CommonTokenStream (ANTLR 4 Runtime (Optimized) 4.7.3 API)













Overview
Package
Class
Use
Tree
Deprecated
Index
Help




Prev Class
Next Class


Frames
No Frames


All Classes






Summary: 
Nested | 
Field | 
Constr | 
Method


Detail: 
Field | 
Constr | 
Method








org.antlr.v4.runtime
Class CommonTokenStream



java.lang.Object


org.antlr.v4.runtime.BufferedTokenStream


org.antlr.v4.runtime.CommonTokenStream









All Implemented Interfaces:
IntStream, TokenStream




public class CommonTokenStream
extends BufferedTokenStream
This class extends BufferedTokenStream with functionality to filter
 token streams to tokens on a particular channel (tokens where
 Token.getChannel() returns a particular value).

 
 This token stream provides access to all tokens by index or when calling
 methods like BufferedTokenStream.getText(). The channel filtering is only used for code
 accessing tokens via the lookahead methods BufferedTokenStream.LA(int), LT(int), and
 LB(int).

 
 By default, tokens are placed on the default channel
 (Token.DEFAULT_CHANNEL), but may be reassigned by using the
 ->channel(HIDDEN) lexer command, or by using an embedded action to
 call Lexer.setChannel(int).
 

 
 Note: lexer rules which use the ->skip lexer command or call
 Lexer.skip() do not produce tokens at all, so input text matched by
 such a rule will not be available as part of the token stream, regardless of
 channel.











Field Summary

Fields 

Modifier and Type
Field and Description


protected int
channel
Specifies the channel to use for filtering tokens.







Fields inherited from class org.antlr.v4.runtime.BufferedTokenStream
fetchedEOF, p, tokens, tokenSource





Fields inherited from interface org.antlr.v4.runtime.IntStream
EOF, UNKNOWN_SOURCE_NAME








Constructor Summary

Constructors 

Constructor and Description


CommonTokenStream(TokenSource tokenSource)
Constructs a new CommonTokenStream using the specified token
 source and the default token channel (Token.DEFAULT_CHANNEL).



CommonTokenStream(TokenSource tokenSource,
                 int channel)
Constructs a new CommonTokenStream using the specified token
 source and filtering tokens to the specified channel.










Method Summary

Methods 

Modifier and Type
Method and Description


protected int
adjustSeekIndex(int i)
Allowed derived classes to modify the behavior of operations which change
 the current stream position by adjusting the target token index of a seek
 operation.



int
getNumberOfOnChannelTokens()
Count EOF just once.



protected Token
LB(int k) 


Token
LT(int k)
Get the Token instance associated with the value returned by
 LA(k).







Methods inherited from class org.antlr.v4.runtime.BufferedTokenStream
consume, fetch, fill, filterForChannel, get, get, getHiddenTokensToLeft, getHiddenTokensToLeft, getHiddenTokensToRight, getHiddenTokensToRight, getSourceName, getText, getText, getText, getText, getTokens, getTokens, getTokens, getTokens, getTokenSource, index, LA, lazyInit, mark, nextTokenOnChannel, previousTokenOnChannel, release, reset, seek, setTokenSource, setup, size, sync





Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait














Field Detail





channel
protected int channel
Specifies the channel to use for filtering tokens.

 
 The default value is Token.DEFAULT_CHANNEL, which matches the
 default channel assigned to tokens created by the lexer.









Constructor Detail





CommonTokenStream
public CommonTokenStream(@NotNull
                 TokenSource tokenSource)
Constructs a new CommonTokenStream using the specified token
 source and the default token channel (Token.DEFAULT_CHANNEL).
Parameters:
tokenSource - The token source.







CommonTokenStream
public CommonTokenStream(@NotNull
                 TokenSource tokenSource,
                 int channel)
Constructs a new CommonTokenStream using the specified token
 source and filtering tokens to the specified channel. Only tokens whose
 Token.getChannel() matches channel or have the
 Token.getType() equal to Token.EOF will be returned by the
 token stream lookahead methods.
Parameters:
tokenSource - The token source.
channel - The channel to use for filtering tokens.









Method Detail





adjustSeekIndex
protected int adjustSeekIndex(int i)
Description copied from class: BufferedTokenStream
Allowed derived classes to modify the behavior of operations which change
 the current stream position by adjusting the target token index of a seek
 operation. The default implementation simply returns i. If an
 exception is thrown in this method, the current stream index should not be
 changed.

 For example, CommonTokenStream overrides this method to ensure that
 the seek target is always an on-channel token.

Overrides:
adjustSeekIndex in class BufferedTokenStream
Parameters:
i - The target token index.
Returns:
The adjusted target token index.







LB
protected Token LB(int k)

Overrides:
LB in class BufferedTokenStream








LT
public Token LT(int k)
Description copied from interface: TokenStream
Get the Token instance associated with the value returned by
 LA(k). This method has the same pre- and post-conditions as
 IntStream.LA(int). In addition, when the preconditions of this method
 are met, the return value is non-null and the value of
 LT(k).getType()==LA(k).

Specified by:
LT in interface TokenStream
Overrides:
LT in class BufferedTokenStream
See Also:
IntStream.LA(int)







getNumberOfOnChannelTokens
public int getNumberOfOnChannelTokens()
Count EOF just once.
















Overview
Package
Class
Use
Tree
Deprecated
Index
Help




Prev Class
Next Class


Frames
No Frames


All Classes






Summary: 
Nested | 
Field | 
Constr | 
Method


Detail: 
Field | 
Constr | 
Method






Copyright © 1992–2019 Tunnel Vision Laboratories, LLC. All rights reserved.

Modifier and Type	Method and Description
`protected int`	`adjustSeekIndex(int i)` Allowed derived classes to modify the behavior of operations which change the current stream position by adjusting the target token index of a seek operation.
`int`	`getNumberOfOnChannelTokens()` Count EOF just once.
`protected Token`	`LB(int k)`
`Token`	`LT(int k)` Get the `Token` instance associated with the value returned by `LA(k)`.