We've been building the same app for over a year with AI agents @atlas_v_erified.
Not one model. A bunch of them. Testing models, QA models, building models, design models. Frontend, backend, schema, architecture — all of it. Cursor keeps shipping new ones (theirs and other labs) and we keep trying them.
And for most of that year period we had no real way to answer a pretty basic question: which model actually wrote this code or which model created this plan?
We had anecdotes.
Take for example Opus 4.7. Promising, but we learned after heavy use that it would say the work was done and only complete part of it. Partial implementations. We figured that out fast and moved to a different model. But that's still just a feeling. Anecdote, not hard data.
So I built model tracking into the repo.
Every time an agent touches a file, it has to leave a byline. Machine-readable. Same session, same sprint.
Here is what we record:
the coding model that made the edit
the planner model, if planning was a different model
the exact hunks that changed
the plan those changes came from
and if it's a bug fix, which earlier change it's fixing (when we actually know)
On a normal TypeScript file it looks like this at the top:
Here are a few rules that matter when implemented:
Newest byline only in the file. Next edit replaces it. Git keeps the history.
Hunks don't include the provenance block itself, or the line numbers get useless the second the header moves.
If Cursor doesn't expose the real model id / max mode / context window, we write not_exposed. Agents are not allowed to invent it. Planner and coder are separate fields on purpose. A lot of our work is planned by one model and implemented by another.
JSON, lockfiles, binaries can't take comments — those go as one line into .cursor/agent-provenance.jsonl. That sidecar is not a loophole for "this .ts file was big."
Docs weren't enough
Now, here is where it gets interesting. First attempt was the schema and a hard rule in our AGENTS.md (yes, I know that prompts are NOT rules or security or requirements, but it's a fast test).
The agents ignored it when it got inconvenient. Left hunks: pending. Skipped headers at the end of large plans and sprints.
So I made it into something they can't skip, and it's working great.
How we force it
Four layers:
Always-on Cursor rule — short reminder every turn
Cursor hooks — track every edit, nag after writes, and on stop force one cleanup turn if anything is missing
Pre-commit — checker blocks the commit
CI — same checker on the PR diff
The stop hook is the one that actually changed behavior.
Agent thinks it's done → we re-check every file from that session → if provenance is missing, it gets forced back in:
PROVENANCE GATE: You edited files without valid atlas-agent-provenance/v1 attribution. Do not start new feature work. Fix provenance on every file below, then stop.
One loop max. Pre-commit and CI catch the rest.
Rules (or .md) are optional to a model. Hooks and CI are not.
Why bother
Once this is all the way through our codebase, we can actually debug where bugs or architectural changes were planned and implement by model.
which coding models create the most rework
which planner → coder combos ship clean on the first pass
when we fix a bug, can we walk it back to the change_id and the model that introduced it
is the "said it was done, only did half" failure mode concentrated in specific models
Publicly available
I really wish I had this data back to the start of this project. It would be incredibly valuable to us. Imagine running a plan to review ALL code generated by a specific model (in loops), or tracing back every issue and running data on which models produce the least issues (even segmenting by back-end, front-end, db schema, etc).
So, I'm going to drop the public repo here so anyone can utilize this, and I have submitted this to Cursor Marketplace for anyone to one-click add (if they approve, 🤞 @cursor_ai).
Drop a comment below with what you are building using Cursor (@cursor_ai). Would love to see what is being built.
Public Repo