I Built a Programming Language in Pure Python — Lexer, Pratt Parser, Bytecode Compiler, Stack VM
A few months ago I decided to tickle at making a programming language, have AI's help... implementation and English translation.
The result is lumen-lang — a small sorta complete programming language in Pure Python, no dependencies, no parser generators, no shortcuts. It has its own syntax, compiles to bytecode, and runs on a stack-based virtual machine I wrote from scratch.
Here's what's inside and why I made the decisions I did.
Source text → Lexer → Token stream → Pratt Parser → AST → Compiler → Bytecode → VM → Output
Every stage is hand-written. That was deliberate — the goal was to understand what's actually happening at each step, not to produce something in the least amount of code.
The scanner is hand-written with line and column tracking. Nothing exotic — you walk the source character by character, emit tokens, handle string escapes and number literals, and report clean errors with position info.
The part people skip: good error messages at the lexer level save enormous debugging pain later. A bare unexpected character with no position is useless. [line 12, col 7] unexpected character: '@' is not.
The parser uses recursive descent with explicit precedence-climbing methods. Each method handles one precedence level and calls the next:
_equality → _comparison → _term → _factor → _unary → _primary
This is sometimes called a Pratt parser (or top-down operator precedence parsing). The key advantage: adding a new operator or precedence level means adding one method and wiring it into the chain. It doesn't require touching a grammar table or regenerating anything.
It's also readable. You can look at _term and immediately see that it handles + and -. No indirection.
Typed node hierarchy covering all language constructs. Every node is a dataclass — no dict-based AST, no stringly-typed anything. The compiler walks the AST with a visitor pattern and emits instructions for each node type.
The compiler tree-walks the AST and emits a compact instruction set into a Chunk — a flat array of opcodes and operands. Constants live in a constant pool indexed by operand.
You can inspect the compiled output for any program:
lumen --disassemble examples/demo.lm
Output for a counter closure:
== make_counter ==
0000 GET_LOCAL 1
0001 CLOSURE 0 <fn next>
0002 GET_LOCAL 3
0003 RETURN
== next ==
0000 GET_UPVALUE 0
0001 CONSTANT 0
0002 ADD
0003 SET_UPVALUE 0
0004 POP
0005 GET_UPVALUE 0
0006 RETURN
The disassembler is one of the most useful parts of this project. When something in the VM behaves wrong, you can look at exactly what the compiler emitted and trace the problem to the right stage.
The VM is stack-based. Each function call pushes a call frame with its own stack window and instruction pointer.
The interesting part is upvalue capture — how closures capture variables from enclosing scopes.
When the compiler sees a function reference a variable from an outer scope, it emits GET_UPVALUE / SET_UPVALUE instructions instead of GET_LOCAL. The VM maintains an upvalue list per closure object. While the enclosing function is still on the stack, upvalues point directly into the stack. When the enclosing function returns, open upvalues are "closed" — their value is copied out of the stack into the upvalue object itself.
This is the mechanism that makes this work correctly:
fn make_counter(start) {
let value = start;
fn next() {
value = value + 1;
return value;
}
return next;
}
let counter = make_counter(10);
print(counter()); // 11
print(counter()); // 12
next captures value from make_counter's scope. After make_counter returns, value no longer exists on the stack — but counter still has a valid reference to it through the closed upvalue.
Getting this right took the most time of any part of the project.
// Fibonacci
fn fib(n) {
if (n < 2) { return n; }
return fib(n - 1) + fib(n - 2);
}
print(fib(10)); // 55
// Lists and dictionaries
let primes = [2, 3, 5, 7, 11];
print(primes[2]); // 5
let person = {"name": "Ada", "field": "computing"};
print(person["name"]); // Ada
Dynamically typed, expression-oriented, C-ish syntax. Closures, recursion, lists, dicts, builtins (print, len, type, clock).
git clone https://github.com/BleedingCodes/lumen-lang.git
cd lumen-lang
python -m lumen # REPL
python -m lumen examples/demo.lm # Run a file
python -m lumen --disassemble examples/demo.lm # Inspect bytecode
No dependencies. Python 3.11+. MIT license.
Repo: github.com/BleedingCodes/lumen-lang
Built by MainbyteLabs — Python tooling for electronics labs, hardware shops, and Linux-based tech teams.