{"slug": "i-built-a-claude-code-skill-that-finds-broken-callers-before-you-deploy", "title": "I Built a Claude Code Skill That Finds Broken Callers Before You Deploy", "summary": "A developer built Test Shield, a tool that detects broken callers before deployment by tracing which callers depend on changed functions. The tool was created after a bug fix in a Chinese chess app silently broke two callers, causing production issues despite 87% test coverage. Test Shield uses AST static analysis to identify impacted callers and generate regression tests, with an honesty report for dynamic call blind spots.", "body_md": "You fix a bug. You deploy. Three hours later, production is on fire. Here's how I built a tool to stop that.\n\nI was working on a Chinese chess app. Changed `is_checkmate()`\n\n— added a simple empty-board guard clause. Two lines. Trivial.\n\nRan my tests. Green. Deployed.\n\nTwo hours later: puzzle solving was broken. Level submission returned wrong results. **I changed one function. It silently broke two callers I didn't even know existed.**\n\nThe test coverage report said 87%. It didn't matter. Coverage tells you which lines ran — not which callers your change silently corrupted.\n\n**Test Shield** — a Claude Code skill with one job:\n\n```\n/test-shield\n     │\n     ▼\nChanged: is_checkmate() — chess_utils.py:320\n\n⚠️  Unexpected impact:\n  ● solve_puzzle() — levels.py:687     ← wouldn't have caught this\n  ● submit_level_solution() — levels.py:516   ← or this\n\n→ Generate 6 regression tests\n→ 4/6 pass, 2 need review\n→ Safe to merge. Here's the evidence.\n```\n\nCoverage tells you: \"Line 42 was executed during tests.\"\n\nTest Shield tells you: \"Line 42 changed. These 3 callers depend on it. They have zero tests. You didn't know about 2 of them.\"\n\nOne is about **what ran**. The other is about **what will break**.\n\nI learned this the hard way during building: Python's dynamic nature means some callers are invisible to static analysis. `getattr()`\n\n, `importlib`\n\n, decorator injection, monkey-patching — AST tracing can't see through those.\n\nSo Test Shield doesn't pretend. Every run ends with a \"honesty report\":\n\n```\nTracing method:         AST static analysis\nDynamic call risks:     0 detected\nKnown blind spots:      getattr, importlib, decorators\n```\n\nIf it can't trace something, it tells you. No false confidence.\n\n```\n# Standalone — no Claude Code required\ngit clone https://github.com/dominicharmon-commits/test-shield.git\ncd your-python-project\npython ../test-shield/scripts/analyze.py .\n\n# With Claude Code\n/test-shield\n```\n\nZero dependencies. Pure Python stdlib. Python 3.10+.\n\nPRs welcome. MIT licensed.\n\n*Built with Claude Code. Tested on a real production codebase. Stars appreciated.* ⭐", "url": "https://wpnews.pro/news/i-built-a-claude-code-skill-that-finds-broken-callers-before-you-deploy", "canonical_source": "https://dev.to/dominic_harmon_18363706c7/i-built-a-claude-code-skill-that-finds-broken-callers-before-you-deploy-52a2", "published_at": "2026-06-27 02:33:31+00:00", "updated_at": "2026-06-27 03:03:59.362698+00:00", "lang": "en", "topics": ["developer-tools", "machine-learning", "ai-tools"], "entities": ["Test Shield", "Claude Code", "GitHub", "Python", "MIT"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-claude-code-skill-that-finds-broken-callers-before-you-deploy", "markdown": "https://wpnews.pro/news/i-built-a-claude-code-skill-that-finds-broken-callers-before-you-deploy.md", "text": "https://wpnews.pro/news/i-built-a-claude-code-skill-that-finds-broken-callers-before-you-deploy.txt", "jsonld": "https://wpnews.pro/news/i-built-a-claude-code-skill-that-finds-broken-callers-before-you-deploy.jsonld"}}