You're iterating on a prompt for an LLM. You tweak one word, the output gets better. You tweak another, it gets worse. You paste it into ChatGPT, it kind of works. You change "concisely" to "in 3 bullets", and suddenly the summaries are perfect.
Then tomorrow comes. And you have no idea which version was the good one.
Your folder looks like this:
prompts/
├── summarize.txt
├── summarize_old.txt
├── summarize_v2.txt
├── summarize_v2_final.txt
├── summarize_v2_final_FINAL.txt
├── summarize_REALLY_final.txt
└── summarize_use_this_one.txt
And the worst part? The "good" one might be any of them. 😭
You can't diff them. You can't tell what changed between v2_final
and v2_final_FINAL
. You can't roll back to the version that actually worked.
Good question. I asked myself the same thing.
You can. And technically, PromptVault borrows git's entire model — content-addressed objects, trees, commits. But in practice, prompts live next to code. They're mixed into repos, notebooks, chat exports, and random .txt
files on your desktop.
Prompts deserve their own version control that:
.txt/.md/.prompt/.j2/.yaml
files specifically.pv/
folder that doesn't collide with your code repo's .git/
So I built it. 🛠️
PromptVault is git
, but purpose-built for prompts. It's written in Rust, ships as a single ~2MB static binary, and runs entirely on your machine. No account, no cloud, no telemetry.
$ pv init
Initialized empty prompt vault in ./.pv
$ pv add prompts/summarize.md
added: prompts/summarize.md
$ pv commit -m "refine: summarize now reports tone + title"
[main 791151d] refine: summarize now reports tone + title
1 prompt
Now iterate. See exactly what changed:
$ pv diff prompts/summarize.md
diff -- prompts/summarize.md
You are a precise summarizer.
-Summarize the text below in 3 concise bullets, then propose a title.
+Summarize the text below in 3 concise bullets, propose a title, and note the tone.
{{text}}
Walk back through every iteration:
$ pv log
commit 791151d…
parent 4100199…
Date: Mon Aug 10 10:04:33 2026 +0000
refine: summarize now reports tone + title
commit 4100199…
Date: Mon Aug 10 10:04:33 2026 +0000
feat: initial prompt set
Restore any past version by hash, tag, or branch:
$ pv show 4100199 # prefix works too 🔍
$ pv revert v1.0 # restore working tree to a tagged version ⏪
This is the killer feature for prompt engineering. You want to test two variants of the same prompt? Branch it.
$ pv branch experiment
$ pv checkout experiment
$ pv checkout main # working tree restores to main's version
$ pv checkout experiment # ...and back to experiment's version
Switch branches and the working tree restores instantly. No copying files, no renaming, no "wait, which folder was the experiment in?" 🎯
When you're done, compare them against a dataset:
$ pv ab main:summarize.md experiment:summarize.md -d cases.jsonl --show
A/B: A=main:summarize.md B=experiment:summarize.md (3 cases)
[1/3] DIFF differs
--- A vs B ---
-A You are a precise summarizer.
+B You are a concise summarizer.
hello world
--- end ---
Summary: 0 identical, 3 differing (of 3)
Pure local. No model calls. No API keys. 🔒
One thing that annoyed me about existing prompt tooling: everything wants to call a model. Every eval run costs money. Every test sends data to OpenAI.
PromptVault takes a different stance: it never calls a model. It only renders templates and checks assertions.
$ pv eval summarize.md --dataset cases.jsonl --show
Eval: summarize.md (3 cases)
[1/3] PASS contains "3 concise bullets"
--- rendered prompt ---
You are a precise summarizer.
...
--- end ---
Summary: 2/2 passed (100%)
You write a JSON Lines dataset, each line fills the prompt's {{variables}}
, and PromptVault renders + asserts. ✅
If you want to actually run the prompt through a model, pipe the rendered output to whatever runner you trust (curl, the OpenAI CLI, ollama, your own script).
This separates two concerns that existing tools conflate:
PromptVault does (1). You choose how to do (2).
For a "tiny git for prompts", it ended up with more than I planned:
pv diff --stat
for a one-line summary per file.pv branch experiment
, pv merge experiment
(fast-forward or three-way, with conflict markers)pv tag v1.0
, pv revert v1.0
.pvignore
pv diff v1 v2
compares any two commits/tags/branchespv show HEAD:summarize.md
reads any file at any refpv tui
launches an interactive commit browserpv push
/ pv pull
syncs the vault to any git host as a backing store--features run
)pv run
against OpenAI/Anthropic/Ollama. pv ab main:x experiment:x -d dataset.jsonl
renders two versions against the same dataset and diffs themPromptVault is a tiny git. On pv init
it creates:
.pv/
├── HEAD → "ref: refs/heads/main"
├── index.json → staging area (path → blob hash)
├── objects/ → content-addressed store (SHA-256)
│ └── ab/cdef… → "<type>\0<data>" (blob / tree / commit)
└── refs/heads/main → latest commit hash
Every prompt version is a blob addressed by the SHA-256 of blob\0<content>
. A tree maps paths to blobs. A commit points to a tree + parent + message. Identical content is stored once. Nothing ever leaves your machine unless you explicitly pv push
. 📦
It uses Myers diff for line-level changes (same algorithm as git), and the glob matcher for .pvignore
is a dynamic-programming implementation to avoid the exponential backtracking that naive recursive matchers hit on patterns like *a*a*a*
. 🧠
Three options:
cargo install promptvault
git clone https://github.com/lululuhu/PromptVault
cd PromptVault
cargo build --release
Then, in any folder where you keep prompts:
pv init
That's it. You're versioning prompts. 🎉
A quick word on this, because I think it matters.
PromptVault is local-first. Everything lives in .pv/
on your machine. No account, no cloud, no telemetry, no analytics, no "phone home". The only network calls are the ones you explicitly make:
pv push
/ pv pull
to sync to a git remote you configuredpv run
(opt-in feature) to send a rendered prompt to a model APIThe pv run
command reads API keys only from environment variables. Nothing is logged. Nothing is stored beyond the rendered prompt itself. 🔐
Important caveat: if your prompts contain secrets, PII, or confidential information, committing them to a vault stores that data on disk in plaintext (content-addressed, but unencrypted). Pushing to a remote sends it to that git host. Same rule as git — don't commit secrets you wouldn't commit to git. ⚠️
This is v0.2.0. The roadmap ahead:
pv mergetool
is planned){{var}}
substitution)If you have opinions, the issue tracker is open. 🚪
If you've ever lost a good prompt to "v2_final_FINAL.txt" syndrome, give it a spin:
cargo install promptvault
⭐ Star it if it's useful.
🐛 Open issues if it's not.
I'm building this in the open and good ideas ship fast.
PromptVault is MIT-licensed, written in Rust 🦀, and runs on Linux/macOS/Windows. The author has no affiliation with any AI lab.