pm_lex_mode Struct Reference

When lexing Ruby source, the lexer has a small amount of state to tell which kind of token it is currently lexing. More...

#include <parser.h>

Data Structures
union	[union].as
	The data associated with this type of lex mode. More...

Public Types
enum	{   PM_LEX_DEFAULT , PM_LEX_EMBEXPR , PM_LEX_EMBVAR , PM_LEX_HEREDOC ,   PM_LEX_LIST , PM_LEX_REGEXP , PM_LEX_STRING }
	The type of this lex mode. More...

Data Fields
enum { ... }	mode
union	as
struct pm_lex_mode *	prev
	The previous lex state so that it knows how to pop.

Detailed Description

When lexing Ruby source, the lexer has a small amount of state to tell which kind of token it is currently lexing.

For example, when we find the start of a string, the first token that we return is a TOKEN_STRING_BEGIN token. After that the lexer is now in the PM_LEX_STRING mode, and will return tokens that are found as part of a string.

Definition at line 109 of file parser.h.

Member Enumeration Documentation

anonymous enum

anonymous enum

The type of this lex mode.

Enumerator
PM_LEX_DEFAULT	This state is used when any given token is being lexed.
PM_LEX_EMBEXPR	This state is used when we're lexing as normal but inside an embedded expression of a string.
PM_LEX_EMBVAR	This state is used when we're lexing a variable that is embedded directly inside of a string with the # shorthand.
PM_LEX_HEREDOC	This state is used when you are inside the content of a heredoc.
PM_LEX_LIST	This state is used when we are lexing a list of tokens, as in a w word list literal or a i symbol list literal.
PM_LEX_REGEXP	This state is used when a regular expression has been begun and we are looking for the terminator.
PM_LEX_STRING	This state is used when we are lexing a string or a string-like token, as in string content with either quote or an xstring.

Definition at line 111 of file parser.h.

Field Documentation

prev

struct pm_lex_mode* pm_lex_mode::prev

The previous lex state so that it knows how to pop.

Definition at line 254 of file parser.h.

---

The documentation for this struct was generated from the following file:

• prism/parser.h