If you use Claude Code heavily, you've probably hit the wall.
You're deep into a session, the agent is finally understanding your codebase, and then — usage limit reached. Not after 5 hours. After 2.
Aside from the limit, a major contributing factor is that every coding session re-sends the same project context over and over. Even with prompt caching, the tokens stack up fast on a large repo, and you burn through your window faster than you'd expect.
So I built L-SDF — Latent-Structured Documentation Format.
What Is L-SDF? #
L-SDF generates compact index files (.lsdf
) that give AI coding agents a structural map of your codebase — before they ever open a source file.
Instead of an agent reading 110K tokens of raw source code to find its way around, it reads an 8K token index first. It only drills into source files when it actually needs implementation details.
The result: significantly reduced input token costs compared with reading source directly, even with prompt caching — which means your usage limit stretches a lot further in navigation-heavy coding sessions.
It works today with Claude Code, Cursor, and Copilot — and because it's a plain file format, it's not tied to any one tool.
The Token Economics #
Here's what the savings look like on a typical Python repo:
| Metric | Value |
|---|---|
| Source tokens (21 files) | 110,432 |
| L-SDF index tokens | 8,241 |
| Compression ratio | 13.4× |
And in terms of estimated input token session cost (output and other costs not included):
| Scenario | Est. 50-turn session cost (input tokens only) |
|---|---|
| Source code, with caching | $2.03 |
| L-SDF + caching | $0.55 |
| Savings | |
| ~73% (~4×) |
How It Works #
L-SDF uses a hierarchical sigil-based topology to represent your codebase structure. Each directory gets two index files:
— shows what exists (compact structural map)INDEX.lsdf
— shows signatures, schemas, and call edgesINDEX.detail.lsdf
The agent reads INDEX.lsdf
first to navigate. It only opens INDEX.detail.lsdf
when it needs signatures or call edges, and opens source files only when it needs the implementation body. This is the key insight: agents don't need to read your source files to understand where things are.
Here's a simple example. Given this Python file:
import sys
class Greeter:
def say_hello(self, name: str) -> str:
if not name:
raise ValueError("Name must not be empty")
message = f"Hello, {name}!"
print(message)
return message
def greet(self, names: list[str]) -> list[str]:
return [self.say_hello(n) for n in names if n.strip()]
L-SDF generates two index files:
** INDEX.lsdf**:
@hello.py
~sys
@Greeter
!say_hello
!greet
** INDEX.detail.lsdf**:
@hello.py
~sys
@Greeter
!say_hello(name:s):s
!greet(names:[s]):[s] > say_hello
In a handful of tokens the agent knows: there's a file hello.py
, it imports sys
, contains a class Greeter
with two methods — their signatures, return types, and that greet
calls say_hello
. No implementation details, no noise — just the map.
Quick Start #
L-SDF is distributed as a global CLI tool. Install it with pipx
so the lsdf
command is available across all your projects:
sudo apt install pipx
pipx ensurepath
pipx install lsdf-core
lsdf --help
Then three commands to set it up in any repo:
lsdf init
lsdf gen . --recursive
lsdf stats
Contributors:If you have the repo checked out locally and want to work on the source, see the[repo README]for editable install and test instructions.
What lsdf init Creates #
project.lsdf
.lsdf/
lsdf_instructions.md
lsdf_spec.md
.lsdfignore
It also detects your stack and skips irrelevant paths (__pycache__
, .venv
, etc.) automatically via .lsdfignore
.
lsdf init
automatically appends instructions to CLAUDE.md
and .cursorrules
so your agents know to use the index files right away.
How It Compares to Other Approaches #
vs. Vector embeddings (Cursor's built-in RAG)
Cursor injects context via vector embeddings — powerful but opaque. You don't control what gets sent or at what cost. L-SDF is an explicit, versionable map in your repo. You can inspect it, version it, and it behaves consistently across every agent. The tradeoff is determinism and zero runtime overhead instead of recall flexibility.
vs. Workflow governance tools (like Haxaml)
Tools like Haxaml govern how the agent works through a task (prepare, build, verify, record). L-SDF governs what the agent reads — giving it a structural map so it navigates without opening files. They're complementary, not competitive. You could run both.
vs. AGENTS.md / CLAUDE.md / .cursorrules
These are prompt files — instructions to the agent. L-SDF is the structured data underneath them. lsdf init
writes to these files automatically so they point agents at the index.
vs. Graphify
Graphify turns your entire project — code, docs, PDFs, images, videos — into a queryable knowledge graph with an interactive HTML visualization. It's a powerful tool for onboarding, architecture exploration, and cross-repo discovery. L-SDF is more narrowly focused: two compact index files per directory, optimized for low-token coding navigation during active sessions. Graphify's output is rich and explorable; L-SDF's output is minimal by design. They're complementary — Graphify for understanding a codebase, L-SDF for navigating it efficiently while coding.
Index Drift #
One concern people raise: what happens when the code changes and the index goes stale?
L-SDF has three layers of defense:
1. Auto-regeneration after structural edits. The agent is instructed to run lsdf gen <dir>
after any structural change. You do the same when editing manually.
2. lsdf sync as an enforcement check. Run it in CI or as a pre-commit hook:
lsdf sync . --check
The exit code is non-zero if any index is out of date. Wire this into your CI's required checks and stale indices stop reaching main
.
3. Auto-regeneration on push via lsdf init --ci. The strongest enforcement — regenerates the index on every push. Requires write permissions on the branch and may add some commit noise, but gives you the highest confidence that the map is always current.
What's Next #
L-SDF currently supports Python repos. The format is language-agnostic and open — generators for Go, Rust, and TypeScript are on the roadmap and contributions are very welcome.
The community has also asked for benchmarks across repo sizes and context thread lengths. Rather than publishing self-generated data, I'd love for early adopters to share their own numbers.
Try It #
- 📦 PyPI: pypi.org/project/lsdf-core - 🔗 GitHub: github.com/ec1980/lsdf-core
If you try it, I'd genuinely love to hear what token savings you see on your repo. Drop a comment below or open a GitHub Discussion.