Why We Built Deterministic AST Refactoring Instead of Prompting LLMs

A developer built TokenCap, a refactoring tool that uses deterministic tree-sitter AST traversal instead of LLM token prediction to perform code transformations such as renaming symbols across a call graph. The tool resolves exact symbol references while skipping string literals and comments, and includes a dead-code pruning command based on an AST call-resolution graph, leaving the LLM to handle only high-level decisions.

When developers ask an AI agent to "rename handleUser to processUser across the project", most agents perform fuzzy grep searches and attempt sequential file rewrites. This approach is prone to edge-case errors: handleUser inside string literals or unrelated comments. In TokenCap, refactoring commands are executed by deterministic tree-sitter AST traversal rather than generative token prediction. Safely rename a function across the entire call graph tokencap refactor rename src/auth.js:validateSession authenticateSession --dry-run Output: Inspecting 58 files... Resolved 12 exact symbol references: ✓ src/auth.js:24: declaration ✓ src/middleware/session.js:15: call site ✓ src/routes/api.js:82: call site - Skipping string literal at src/tests/mock.js:12 "validateSession" Dry run complete: 12 references updated across 3 files with 0 syntax errors. Similarly, tokencap refactor rm-dead uses our AST call-resolution graph to detect uncalled, unexported dead functions and prune them safely. By using the LLM for high-level decision making and TokenCap for deterministic syntax-tree transforms, refactor operations remain safe and reproducible. Documentation available at tokencap.vansharora.app