{"slug": "the-tracking-plan-said-implemented-the-dashboard-says-otherwise-i-built-a-cli", "title": "The tracking plan said \"implemented\". The dashboard says otherwise. I built a CLI that detects plan↔code drift using only AST.", "summary": "A developer has released plan-drift, an open-source CLI tool that detects drift between an analytics tracking plan and its actual code implementation using only Python's standard ast module, with no LLM involved. The tool cross-references a JSON tracking plan against track() calls in a repository to flag property mismatches, unexpected events, and unimplemented events in both directions. The developer argues deterministic work deserves deterministic tools, prioritizing accuracy and reproducibility over LLM-based parsing.", "body_md": "\"Did shipping auth tracking make it into the release?\"\n\n\"Yep, it's in.\"\n\nTwo weeks later, someone opens the dashboard and says \"…this doesn't look right.\"\n\nThis is about **plan-drift**, a CLI I built that detects exactly that gap — the silent drift between your analytics **tracking plan** (what you intended to measure) and the **actual implementation** — **without using an LLM at all**.\n\n[https://github.com/sunnydachs/plan-drift](https://github.com/sunnydachs/plan-drift)\n\nYou give it a repository and a tracking plan (a JSON file). It finds four kinds of drift:\n\n`track()` call found\n\n```\n# scan the current repo against your plan (read-only)\nplan-drift --plan tracking-plan.json\n\n# another path, JSON output\nplan-drift --plan tracking-plan.json ./src --json\n```\n\nOutput looks like this:\n\n```\nplan-drift — scanned /home/dev/myproject\n  tracking plan events: 4 | track() calls found: 6 (dynamic: 1)\napp/events.py:12  [!] PROPERTY MISMATCH\n    event 'Signed Up': undeclared property 'campaign' found in code but not in plan\napp/analytics.py:34  [+] UNEXPECTED EVENT\n    'Add To Cart' implemented (1 call) but not in the tracking plan\napp/analytics.py:56  [ ] UNIMPLEMENTED EVENT\n    'Checkout Started' declared in plan but no track() call found\n```\n\n\"The event shipped but the properties are off\" is invisible to IDEs and static analysis. Tracking code is rarely covered by tests. So nobody notices until the numbers look wrong.\n\nThis was the core design decision. LLMs could parse ambiguous tracking code with more \"understanding.\" But I built it on **nothing but Python's standard `ast` (Abstract Syntax Tree)**. Three reasons:\n\n`pip install git+...` and you're done.\nTracking calls like `analytics.track(...)` are statically analyzable — an LLM here would add cost, latency, and nondeterminism with no accuracy gain.\n\nThis is one answer to a question I keep coming back to when automating: *how much should AI handle?* Deterministic work deserves deterministic tools.\n\nThe thing I cared most about was **checking plan↔implementation in both directions**:\n\nSo the tool cross-references and enumerates **every mismatch**. Test files (`tests.py`, `test_*.py`) are excluded, because fixture events showing up as \"unplanned implementations\" would be pure noise.\n\nThe \"plan exists but implementation never got updated\" problem I've seen at workplaces — this gets you most of the way there.\n\n`track(event_name, props)` shows up as `DYNAMIC` for manual review.`.py`). JS needs a different parser — planned.` plan: string` and the code sends an int, that's a future improvement.\nThese are documented on the roadmap. The current version prioritizes correctness and low false positives over coverage.\n\nDrift between planning and implementation quietly corrupts your dashboards. **plan-drift** fights it with a read-only, fully deterministic, LLM-free design — accuracy and reproducibility first.\n\nHappy to hear any feedback — whether you're deploying it in-house or just trying it out. It's on GitHub.\n\n*This is a personal OSS project with no warranty. If you hit bugs or have suggestions, GitHub issues are the best way to reach me.*", "url": "https://wpnews.pro/news/the-tracking-plan-said-implemented-the-dashboard-says-otherwise-i-built-a-cli", "canonical_source": "https://dev.to/sunnydachs/the-tracking-plan-said-implemented-the-dashboard-says-otherwise-i-built-a-cli-that-detects-2b8a", "published_at": "2026-09-18 02:09:41+00:00", "updated_at": "2026-09-18 02:52:59.910019+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools"], "entities": ["plan-drift", "GitHub", "Python"], "alternates": {"html": "https://wpnews.pro/news/the-tracking-plan-said-implemented-the-dashboard-says-otherwise-i-built-a-cli", "markdown": "https://wpnews.pro/news/the-tracking-plan-said-implemented-the-dashboard-says-otherwise-i-built-a-cli.md", "text": "https://wpnews.pro/news/the-tracking-plan-said-implemented-the-dashboard-says-otherwise-i-built-a-cli.txt", "jsonld": "https://wpnews.pro/news/the-tracking-plan-said-implemented-the-dashboard-says-otherwise-i-built-a-cli.jsonld"}}