Package org.stringtemplate.v4.compiler
Class STLexer
- java.lang.Object
-
- org.stringtemplate.v4.compiler.STLexer
-
- All Implemented Interfaces:
org.antlr.runtime.TokenSource
public class STLexer extends Object implements org.antlr.runtime.TokenSource
This class represents the tokenizer for templates. It operates in two modes: inside and outside of expressions. It implements theTokenSource
interface so it can be used with ANTLR parsers. Outside of expressions, we can return these token types:TEXT
,INDENT
,LDELIM
(start of expression),RCURLY
(end of subtemplate), andNEWLINE
. Inside of an expression, this lexer returns all of the tokens needed bySTParser
. From the parser's point of view, it can treat a template as a simple stream of elements.This class defines the token types and communicates these values to
STParser.g
viaSTLexer.tokens
file (which must remain consistent).
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
STLexer.STToken
We buildSTToken
tokens instead of relying onCommonToken
so we can overrideSTLexer.STToken.toString()
.
-
Field Summary
Fields Modifier and Type Field Description static int
AND
static int
AT
static int
BANG
static int
COLON
static int
COMMA
static int
COMMENT
static int
DOT
static int
ELLIPSIS
static int
ELSE
static int
ELSEIF
static int
ENDIF
static char
EOF
static int
EOF_TYPE
static int
EQUALS
static int
FALSE
static int
ID
static int
IF
static int
INDENT
static int
LBRACK
static int
LCURLY
static int
LDELIM
static int
LPAREN
static int
NEWLINE
static int
OR
static int
PIPE
static int
RBRACK
static int
RCURLY
static int
RDELIM
static int
REGION_END
static int
RPAREN
static int
SEMI
static org.antlr.runtime.Token
SKIP
static int
SLASH
static int
STRING
int
subtemplateDepth
To be able to properly track the inside/outside mode, we need to track how deeply nested we are in some templates.static int
SUPER
static int
TEXT
static int
TRUE
-
Constructor Summary
Constructors Constructor Description STLexer(org.antlr.runtime.CharStream input)
STLexer(ErrorManager errMgr, org.antlr.runtime.CharStream input, org.antlr.runtime.Token templateToken)
STLexer(ErrorManager errMgr, org.antlr.runtime.CharStream input, org.antlr.runtime.Token templateToken, char delimiterStartChar, char delimiterStopChar)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description org.antlr.runtime.Token
_nextToken()
protected void
consume()
void
emit(org.antlr.runtime.Token token)
String
getSourceName()
protected org.antlr.runtime.Token
inside()
static boolean
isIDLetter(char c)
static boolean
isIDStartLetter(char c)
static boolean
isUnicodeLetter(char c)
static boolean
isWS(char c)
void
match(char x)
Consume ifx
is next character on the input stream.org.antlr.runtime.Token
newToken(int ttype)
org.antlr.runtime.Token
newToken(int ttype, String text)
org.antlr.runtime.Token
newToken(int ttype, String text, int pos)
org.antlr.runtime.Token
newTokenFromPreviousChar(int ttype)
org.antlr.runtime.Token
nextToken()
protected org.antlr.runtime.Token
outside()
static String
str(int c)
-
-
-
Field Detail
-
EOF
public static final char EOF
- See Also:
- Constant Field Values
-
EOF_TYPE
public static final int EOF_TYPE
- See Also:
- Constant Field Values
-
SKIP
public static final org.antlr.runtime.Token SKIP
-
RBRACK
public static final int RBRACK
- See Also:
- Constant Field Values
-
LBRACK
public static final int LBRACK
- See Also:
- Constant Field Values
-
ELSE
public static final int ELSE
- See Also:
- Constant Field Values
-
ELLIPSIS
public static final int ELLIPSIS
- See Also:
- Constant Field Values
-
LCURLY
public static final int LCURLY
- See Also:
- Constant Field Values
-
BANG
public static final int BANG
- See Also:
- Constant Field Values
-
EQUALS
public static final int EQUALS
- See Also:
- Constant Field Values
-
TEXT
public static final int TEXT
- See Also:
- Constant Field Values
-
ID
public static final int ID
- See Also:
- Constant Field Values
-
SEMI
public static final int SEMI
- See Also:
- Constant Field Values
-
LPAREN
public static final int LPAREN
- See Also:
- Constant Field Values
-
IF
public static final int IF
- See Also:
- Constant Field Values
-
ELSEIF
public static final int ELSEIF
- See Also:
- Constant Field Values
-
COLON
public static final int COLON
- See Also:
- Constant Field Values
-
RPAREN
public static final int RPAREN
- See Also:
- Constant Field Values
-
COMMA
public static final int COMMA
- See Also:
- Constant Field Values
-
RCURLY
public static final int RCURLY
- See Also:
- Constant Field Values
-
ENDIF
public static final int ENDIF
- See Also:
- Constant Field Values
-
RDELIM
public static final int RDELIM
- See Also:
- Constant Field Values
-
SUPER
public static final int SUPER
- See Also:
- Constant Field Values
-
DOT
public static final int DOT
- See Also:
- Constant Field Values
-
LDELIM
public static final int LDELIM
- See Also:
- Constant Field Values
-
STRING
public static final int STRING
- See Also:
- Constant Field Values
-
PIPE
public static final int PIPE
- See Also:
- Constant Field Values
-
OR
public static final int OR
- See Also:
- Constant Field Values
-
AND
public static final int AND
- See Also:
- Constant Field Values
-
INDENT
public static final int INDENT
- See Also:
- Constant Field Values
-
NEWLINE
public static final int NEWLINE
- See Also:
- Constant Field Values
-
AT
public static final int AT
- See Also:
- Constant Field Values
-
REGION_END
public static final int REGION_END
- See Also:
- Constant Field Values
-
TRUE
public static final int TRUE
- See Also:
- Constant Field Values
-
FALSE
public static final int FALSE
- See Also:
- Constant Field Values
-
COMMENT
public static final int COMMENT
- See Also:
- Constant Field Values
-
SLASH
public static final int SLASH
- See Also:
- Constant Field Values
-
subtemplateDepth
public int subtemplateDepth
To be able to properly track the inside/outside mode, we need to track how deeply nested we are in some templates. Otherwise, we know whether a'}'
and the outermost subtemplate to send this back to outside mode.
-
-
Constructor Detail
-
STLexer
public STLexer(org.antlr.runtime.CharStream input)
-
STLexer
public STLexer(ErrorManager errMgr, org.antlr.runtime.CharStream input, org.antlr.runtime.Token templateToken)
-
STLexer
public STLexer(ErrorManager errMgr, org.antlr.runtime.CharStream input, org.antlr.runtime.Token templateToken, char delimiterStartChar, char delimiterStopChar)
-
-
Method Detail
-
nextToken
public org.antlr.runtime.Token nextToken()
- Specified by:
nextToken
in interfaceorg.antlr.runtime.TokenSource
-
match
public void match(char x)
Consume ifx
is next character on the input stream.
-
consume
protected void consume()
-
emit
public void emit(org.antlr.runtime.Token token)
-
_nextToken
public org.antlr.runtime.Token _nextToken()
-
outside
protected org.antlr.runtime.Token outside()
-
inside
protected org.antlr.runtime.Token inside()
-
isIDStartLetter
public static boolean isIDStartLetter(char c)
-
isIDLetter
public static boolean isIDLetter(char c)
-
isWS
public static boolean isWS(char c)
-
isUnicodeLetter
public static boolean isUnicodeLetter(char c)
-
newToken
public org.antlr.runtime.Token newToken(int ttype)
-
newTokenFromPreviousChar
public org.antlr.runtime.Token newTokenFromPreviousChar(int ttype)
-
newToken
public org.antlr.runtime.Token newToken(int ttype, String text, int pos)
-
newToken
public org.antlr.runtime.Token newToken(int ttype, String text)
-
getSourceName
public String getSourceName()
- Specified by:
getSourceName
in interfaceorg.antlr.runtime.TokenSource
-
str
public static String str(int c)
-
-