A Deep Dive into the Expansion Syntax
A detailed look at how the parsing works and the specific rules for the syntax.
This page goes into a bit more detail about how envfmt parses your template
strings. The whole thing is done in a single pass, which makes it pretty fast.
I’m using a peekable iterator over the string’s characters, which lets me look
ahead one character without consuming it.
The Main Loop
The parser’s main loop goes through the string one character at a time. Most
characters are just added to the output directly. The interesting part happens
when it finds a $ character.
When it sees a $, it peeks at the next character to decide what to do.
- If the next character is another
$, it’s an escape sequence. - If the next character is
{, it starts parsing a braced variable. - If the next character is a letter or an underscore, it starts parsing a simple variable.
- If it’s anything else (like a space, a number, or the end of the string),
the
$is just treated as a literal character.
How $$ is Parsed
This is the easiest case. When the parser sees $$, it just adds a single $
to the output and moves on, consuming both characters from the input.
How $VAR is Parsed
When the parser sees a $ followed by a valid starting character (a-z, A-Z, or
_), it enters a “greedy” loop. It keeps reading characters as long as they are
valid for a variable name (alphanumeric or _).
For example, in a string like $VAR1_is_set, the parser will match the entire
VAR1_is_set as the variable name. It doesn’t stop at VAR1.
Once it hits a character that’s not valid for a variable name (like the space
after is_set), it stops and looks up the name it collected.
How ${VAR} and ${VAR:-default} are Parsed
Braced variables are the most complex part. When the parser sees ${, it starts
collecting characters for the variable name.
It keeps reading until it finds either a } or :-.
- If it finds
}: The variable name is complete. The parser looks up the name in the context. If it’s found, the value is added to the output. If not, it returns aVariableNotFounderror. - If it finds
:-: This means there’s a default value.- First, the parser looks up the variable name it has collected so far (the
part before
:-). - If the variable is found in the context, its value is used. The parser
then has to fast-forward and discard the default value part. It does this
by scanning ahead until it finds the matching closing
}. It even keeps track of nested braces, so a default like${VAR:-{ "key": "val" }}works correctly. - If the variable is not found, the parser starts collecting the default
value. It reads everything until it finds the matching closing
}(again, handling nested braces) and uses that as the value.
- First, the parser looks up the variable name it has collected so far (the
part before
If the parser gets to the end of the string without finding a closing }, it
returns an UnclosedBrace error.
That’s pretty much all the logic. It’s a simple state machine that handles the few syntax rules it supports.