"Did shipping auth tracking make it into the release?"
"Yep, it's in."
Two weeks later, someone opens the dashboard and says "…this doesn't look right."
This is about plan-drift, a CLI I built that detects exactly that gap — the silent drift between your analytics tracking plan (what you intended to measure) and the actual implementation — without using an LLM at all.
https://github.com/sunnydachs/plan-drift
You give it a repository and a tracking plan (a JSON file). It finds four kinds of drift:
track() call found
plan-drift --plan tracking-plan.json
plan-drift --plan tracking-plan.json ./src --json
Output looks like this:
plan-drift — scanned /home/dev/myproject
tracking plan events: 4 | track() calls found: 6 (dynamic: 1)
app/events.py:12 [!] PROPERTY MISMATCH
event 'Signed Up': undeclared property 'campaign' found in code but not in plan
app/analytics.py:34 [+] UNEXPECTED EVENT
'Add To Cart' implemented (1 call) but not in the tracking plan
app/analytics.py:56 [ ] UNIMPLEMENTED EVENT
'Checkout Started' declared in plan but no track() call found
"The event shipped but the properties are off" is invisible to IDEs and static analysis. Tracking code is rarely covered by tests. So nobody notices until the numbers look wrong.
This was the core design decision. LLMs could parse ambiguous tracking code with more "understanding." But I built it on nothing but Python's standard ast (Abstract Syntax Tree). Three reasons:
pip install git+... and you're done.
Tracking calls like analytics.track(...) are statically analyzable — an LLM here would add cost, latency, and nondeterminism with no accuracy gain.
This is one answer to a question I keep coming back to when automating: how much should AI handle? Deterministic work deserves deterministic tools.
The thing I cared most about was checking plan↔implementation in both directions:
So the tool cross-references and enumerates every mismatch. Test files (tests.py, test_*.py) are excluded, because fixture events showing up as "unplanned implementations" would be pure noise.
The "plan exists but implementation never got updated" problem I've seen at workplaces — this gets you most of the way there.
track(event_name, props) shows up as DYNAMIC for manual review..py). JS needs a different parser — planned. plan: string and the code sends an int, that's a future improvement.
These are documented on the roadmap. The current version prioritizes correctness and low false positives over coverage.
Drift between planning and implementation quietly corrupts your dashboards. plan-drift fights it with a read-only, fully deterministic, LLM-free design — accuracy and reproducibility first.
Happy to hear any feedback — whether you're deploying it in-house or just trying it out. It's on GitHub.
This is a personal OSS project with no warranty. If you hit bugs or have suggestions, GitHub issues are the best way to reach me.