{"slug": "why-we-built-deterministic-ast-refactoring-instead-of-prompting-llms", "title": "Why We Built Deterministic AST Refactoring Instead of Prompting LLMs", "summary": "A developer built TokenCap, a refactoring tool that uses deterministic tree-sitter AST traversal instead of LLM token prediction to perform code transformations such as renaming symbols across a call graph. The tool resolves exact symbol references while skipping string literals and comments, and includes a dead-code pruning command based on an AST call-resolution graph, leaving the LLM to handle only high-level decisions.", "body_md": "When developers ask an AI agent to \"rename `handleUser` to `processUser` across the project\", most agents perform fuzzy grep searches and attempt sequential file rewrites.\n\nThis approach is prone to edge-case errors:\n\n`handleUser` inside string literals or unrelated comments.\nIn TokenCap, refactoring commands are executed by deterministic tree-sitter AST traversal rather than generative token prediction.\n\n```\n# Safely rename a function across the entire call graph\ntokencap refactor rename src/auth.js:validateSession authenticateSession --dry-run\n```\n\nOutput:\n\n```\nInspecting 58 files...\nResolved 12 exact symbol references:\n  ✓ src/auth.js:24: declaration\n  ✓ src/middleware/session.js:15: call site\n  ✓ src/routes/api.js:82: call site\n  - Skipping string literal at src/tests/mock.js:12 (\"validateSession\")\nDry run complete: 12 references updated across 3 files with 0 syntax errors.\n```\n\nSimilarly, `tokencap refactor rm-dead` uses our AST call-resolution graph to detect uncalled, unexported dead functions and prune them safely.\n\nBy using the LLM for high-level decision making and TokenCap for deterministic syntax-tree transforms, refactor operations remain safe and reproducible.\n\nDocumentation available at tokencap.vansharora.app", "url": "https://wpnews.pro/news/why-we-built-deterministic-ast-refactoring-instead-of-prompting-llms", "canonical_source": "https://dev.to/vansharora21/why-we-built-deterministic-ast-refactoring-instead-of-prompting-llms-1096", "published_at": "2026-09-30 18:31:42+00:00", "updated_at": "2026-09-30 18:46:47.725941+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "ai-tools"], "entities": ["TokenCap", "tree-sitter"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/why-we-built-deterministic-ast-refactoring-instead-of-prompting-llms", "markdown": "https://wpnews.pro/news/why-we-built-deterministic-ast-refactoring-instead-of-prompting-llms.md", "text": "https://wpnews.pro/news/why-we-built-deterministic-ast-refactoring-instead-of-prompting-llms.txt", "jsonld": "https://wpnews.pro/news/why-we-built-deterministic-ast-refactoring-instead-of-prompting-llms.jsonld"}}