A Deep Dive into the Expansion Syntax

This page goes into a bit more detail about how envfmt parses your template strings. The whole thing is done in a single pass, which makes it pretty fast. I’m using a peekable iterator over the string’s characters, which lets me look ahead one character without consuming it.

The Main Loop

The parser’s main loop goes through the string one character at a time. Most characters are just added to the output directly. The interesting part happens when it finds a $ character.

When it sees a $, it peeks at the next character to decide what to do.

1. If the next character is another $, it’s an escape sequence.
2. If the next character is {, it starts parsing a braced variable.
3. If the next character is a letter or an underscore, it starts parsing a simple variable.
4. If it’s anything else (like a space, a number, or the end of the string), the $ is just treated as a literal character.

How $$ is Parsed

This is the easiest case. When the parser sees $$, it just adds a single $ to the output and moves on, consuming both characters from the input.

How $VAR is Parsed

When the parser sees a $ followed by a valid starting character (a-z, A-Z, or _), it enters a “greedy” loop. It keeps reading characters as long as they are valid for a variable name (alphanumeric or _).

For example, in a string like $VAR1_is_set, the parser will match the entire VAR1_is_set as the variable name. It doesn’t stop at VAR1.

Once it hits a character that’s not valid for a variable name (like the space after is_set), it stops and looks up the name it collected.

How ${VAR} and ${VAR:-default} are Parsed

Braced variables are the most complex part. When the parser sees ${, it starts collecting characters for the variable name.

It keeps reading until it finds either a } or :-.

• If it finds }: The variable name is complete. The parser looks up the name in the context. If it’s found, the value is added to the output. If not, it returns a VariableNotFound error.
• If it finds :-: This means there’s a default value.
  • First, the parser looks up the variable name it has collected so far (the part before :-).
  • If the variable is found in the context, its value is used. The parser then has to fast-forward and discard the default value part. It does this by scanning ahead until it finds the matching closing }. It even keeps track of nested braces, so a default like ${VAR:-{ "key": "val" }} works correctly.
  • If the variable is not found, the parser starts collecting the default value. It reads everything until it finds the matching closing } (again, handling nested braces) and uses that as the value.

If the parser gets to the end of the string without finding a closing }, it returns an UnclosedBrace error.

That’s pretty much all the logic. It’s a simple state machine that handles the few syntax rules it supports.