Why We Built Deterministic AST Refactoring Instead of Prompting LLMs

When developers ask an AI agent to "rename handleUser to processUser across the project", most agents perform fuzzy grep searches and attempt sequential file rewrites.

This approach is prone to edge-case errors:

handleUser inside string literals or unrelated comments. In TokenCap, refactoring commands are executed by deterministic tree-sitter AST traversal rather than generative token prediction.

tokencap refactor rename src/auth.js:validateSession authenticateSession --dry-run

Output:

Inspecting 58 files...
Resolved 12 exact symbol references:
  ✓ src/auth.js:24: declaration
  ✓ src/middleware/session.js:15: call site
  ✓ src/routes/api.js:82: call site
  - Skipping string literal at src/tests/mock.js:12 ("validateSession")
Dry run complete: 12 references updated across 3 files with 0 syntax errors.

Similarly, tokencap refactor rm-dead uses our AST call-resolution graph to detect uncalled, unexported dead functions and prune them safely.

By using the LLM for high-level decision making and TokenCap for deterministic syntax-tree transforms, refactor operations remain safe and reproducible.

Documentation available at tokencap.vansharora.app