{"slug": "from-you-have-a-bug-to-here-s-the-root-cause-adding-ai-code-analysis-to-my-app", "title": "From \"You Have a Bug\" to \"Here's the Root Cause\" - Adding AI Code Analysis to My App Review Pipeline", "summary": "A developer built AppPulse, a CLI tool that classifies app reviews and crash data, then extended it with an AI agent using PydanticAI to identify root causes and suggest code changes. The agent uses read-only tools to search, read, and list files in the codebase, producing a structured AnalysisBrief. The system supports multiple LLM backends including OpenAI, Anthropic, and Ollama.", "body_md": "TL;DR:I built an app review pipeline that classifies bugs and crashes. It was useful but stopped at \"what's wrong.\" I extended it to answer \"where in the code\" and \"what to change\" — using a PydanticAI agent with codebase exploration tools, then made the analysis engine pluggable so it can use Grok Build, Claude Code, or OpenAI Codex as backends. Here's how I got there, what broke along the way, and how the architecture ended up.\n\nI have a CLI tool called **AppPulse** that monitors my app's reviews and crash data. Every morning it:\n\nPulls new reviews from Google Play / App Store\n\nClassifies each one with an LLM — bug, crash, feature request, performance, praise\n\nCorrelates reviews with crash data from Sentry/Firebase\n\nSends me a digest\n\n```\napppulse run\n# → \"12 new reviews: 3 bugs (1 critical), 2 feature requests, 7 praise\"\n\napppulse reviews --category bug\n#  ID │ Rating │ Summary                              │ Severity\n#  42 │  ★☆☆☆☆ │ Photo upload crashes for >10MB images │ critical\n```\n\nThis was genuinely useful. I knew *what* users were complaining about and which crashes were affecting the most people. But the pipeline stopped at classification. When I saw \"Photo upload crashes for >10MB images,\" I still had to:\n\nOpen the project in my IDE\n\nSearch for upload-related code\n\nCross-reference the stacktrace from Sentry\n\nMentally map the review to the actual code\n\nFigure out what to change\n\nFor one bug, that's fine. For a batch of 5 critical bugs on a Monday morning, it's a lot of manual work before I've even started fixing anything.\n\nI wanted the pipeline to go further — to take the classified review, point to the specific files and lines causing the issue, and hand me a brief I could act on immediately.\n\nThe core idea was simple: give an LLM agent access to my codebase through read-only tools and let it investigate the bug, just like I would.\n\nI considered LangGraph, LangChain, and rolling my own agent loop. PydanticAI won for three reasons:\n\n**Structured output by default.** I needed the analysis to come back as a validated Pydantic model — not free-form text I'd have to parse. PydanticAI auto-validates the LLM output against the schema and retries with correction instructions if it's wrong.\n\n**Minimal dependency footprint.** AppPulse is a lightweight CLI. I didn't want to drag in LangChain's dependency tree for a single-agent linear workflow.\n\n**It already supports the LLM providers I use** — OpenAI, Anthropic, Ollama. No new API keys.\n\nEvery analysis produces a validated `AnalysisBrief`\n\n:\n\n```\nclass AnalysisBrief(BaseModel):\n    issue_type: str              # bug | crash | feature_request\n    summary: str                 # one-paragraph assessment\n    root_cause: str | None       # for bugs/crashes\n    feature_rationale: str | None # for feature requests\n    affected_files: list[AffectedFile]\n    proposed_changes: list[ProposedChange]\n    complexity: str              # small | medium | large\n    risks: list[str]             # potential side effects\n    testing_notes: str           # what tests to write/update\n```\n\nThis is the contract between the analysis engine and the rest of the pipeline. Whether the analysis runs in-process or via an external coding agent, it always produces the same structure.\n\nThe agent gets four tools — all read-only, all operating on the local codebase:\n\n``` python\n@analysis_agent.tool\nasync def tool_search_code(ctx: RunContext[AnalysisDeps], keyword: str, file_extensions: str = \"\") -> str:\n    \"\"\"grep/ripgrep search — finds where relevant code lives.\"\"\"\n    return search_code(ctx.deps.code_path, keyword, file_extensions)\n\n@analysis_agent.tool\nasync def tool_read_file(ctx: RunContext[AnalysisDeps], file_path: str, start_line: int = 0, end_line: int = 0) -> str:\n    \"\"\"Read source with line range, capped at 500 lines, path-validated.\"\"\"\n    return read_file(ctx.deps.code_path, file_path, start_line, end_line)\n\n@analysis_agent.tool\nasync def tool_list_files(ctx: RunContext[AnalysisDeps], directory: str = \".\", pattern: str = \"\") -> str:\n    \"\"\"Directory listing with sizes.\"\"\"\n    return list_files(ctx.deps.code_path, directory, pattern)\n\n@analysis_agent.tool\nasync def tool_find_symbols(ctx: RunContext[AnalysisDeps], name: str) -> str:\n    \"\"\"Regex-based definition search — classes, functions, methods.\"\"\"\n    return find_symbols(ctx.deps.code_path, name)\n```\n\nEvery tool validates paths against `code_path`\n\nto prevent directory traversal — no `../../etc/passwd`\n\ngames. The agent can explore but never escape the project root.\n\nThe first version worked, but the agent was expensive. It would make 40-50 tool calls — listing directories, searching for broad terms, reading entire files — before settling on a hypothesis. At `gpt-4o-mini`\n\nrates that's still cheap, but it hit the API request limit I'd set and sometimes never produced output.\n\nThe biggest waste was the agent discovering project structure. It would call `list_files`\n\nrecursively, trying to understand where things lived before it could even start investigating.\n\nThe fix: generate a **repo map** upfront and include it in the system prompt. The repo map is a compact text overview of every source file with its top-level symbols (classes, functions) extracted via regex:\n\n```\nsrc/services/\n  ImageUploadService.java — ImageUploadService, compress, validateSize, uploadToS3\n  UserService.java — UserService, getProfile, updatePreferences\nsrc/utils/\n  ImageUtils.java — ImageUtils, downscale, getExifOrientation\n```\n\nThis gives the agent a \"GPS\" before it makes any tool calls. Instead of blindly listing directories, it can read the repo map and go straight to `ImageUploadService.java`\n\nwhen investigating an upload crash.\n\nGenerating the repo map means walking the entire source tree and reading every file for definitions. For a medium project (~500 files), that's 100-500ms. Fine for a one-off, but I didn't want to pay that cost on every analysis.\n\nThe cache strategy uses a git-based fingerprint:\n\n``` php\ndef _compute_fingerprint(root: Path) -> str:\n    # Git repos: HEAD commit + dirty-file count (~5ms)\n    head = subprocess.run([\"git\", \"rev-parse\", \"HEAD\"], ...)\n    status = subprocess.run([\"git\", \"status\", \"--porcelain\"], ...)\n    dirty_count = len(status.stdout.strip().splitlines())\n    return hashlib.sha256(f\"{commit}:{dirty_count}\".encode()).hexdigest()[:16]\n```\n\nCache hit: ~0ms. Cache miss (new commit or uncommitted changes): regenerate. The cache self-invalidates on every git operation that changes the working tree.\n\nEven with the repo map, the agent would sometimes keep exploring forever — finding tangential files, searching for related patterns, going down rabbit holes. Sound familiar? It's what I do when debugging at 2am.\n\nThe fix is a **funnel** that progressively restricts the agent's tools:\n\n``` python\nasync def _prepare_tools(ctx: RunContext[AnalysisDeps], tool_defs: list[ToolDefinition]):\n    tool_call_count = count_tool_calls(ctx.messages)\n\n    if tool_call_count >= 18:\n        return []           # No tools — force the agent to produce output\n\n    if tool_call_count >= 12:\n        # Only read_file — no more searching, refine what you already found\n        return [td for td in tool_defs if td.name not in EXPLORATION_TOOLS]\n\n    return tool_defs        # Full access during exploration phase\n```\n\n| Phase | Tool Calls | Available Tools |\n|---|---|---|\n| Exploration | 0-12 | All 4 tools |\n| Deep-dive | 12-18 |\n`read_file` only |\n| Forced output | 18+ | None |\n\nThis dropped the average analysis from 40+ tool calls to 8-12, with no noticeable quality difference. The agent just gets to the point faster.\n\nThe in-process PydanticAI agent worked well for straightforward bugs. But for complex, multi-file issues — or when I wanted deeper analysis — I kept wishing I could point a full coding agent like Grok Build or Claude Code at the codebase and just get back a structured brief.\n\nAll of these tools support headless mode:\n\n```\n# Grok Build\ngrok -p \"analyze this bug...\" --cwd /path/to/code --tools \"read_file,grep,list_dir\" --yolo\n\n# Claude Code\nclaude -p \"analyze this bug...\" --cwd /path/to/code --dangerously-skip-permissions\n\n# OpenAI Codex\ncodex --quiet --approval-mode full-auto \"analyze this bug...\"\n```\n\nThe challenge: each agent returns its output differently (JSON, plain text, structured conversations), and I still needed a validated `AnalysisBrief`\n\nat the end.\n\nThe solution is a two-stage pipeline for external backends:\n\n```\nStage 1: Coding Agent → Raw Markdown\nStage 2: Cheap LLM (structurer) → Validated AnalysisBrief\n```\n\n**Stage 1** runs the coding agent with a prompt that asks it to produce a markdown analysis with specific sections (Summary, Root Cause, Affected Files, Proposed Changes, etc.). The agent uses its own tools to explore the codebase — it's much better at this than my four built-in tools because it has full access to grep, file reading, and code understanding.\n\n**Stage 2** is a single, cheap `gpt-4o-mini`\n\ncall that extracts the markdown into the `AnalysisBrief`\n\nPydantic model. No tools, no exploration — just structured extraction. Costs fractions of a cent.\n\nThe raw markdown from Stage 1 is saved as an inspectable artifact at `~/.apppulse/analyses/raw/`\n\n. If the structured output looks wrong, I can check exactly what the coding agent found.\n\nI wanted to be able to add new backends without touching the core analysis flow. The architecture uses a lazy-loaded registry with a strategy pattern:\n\n```\n# backends/__init__.py\nBACKEND_REGISTRY: dict[str, str] = {\n    \"builtin\":     \"apppulse.code_analysis.backends.builtin:BuiltinBackend\",\n    \"grok\":        \"apppulse.code_analysis.backends.grok:GrokBackend\",\n    \"claude_code\": \"apppulse.code_analysis.backends.claude_code:ClaudeCodeBackend\",\n    \"codex\":       \"apppulse.code_analysis.backends.codex:CodexBackend\",\n}\n\ndef get_backend(name: str) -> BaseAnalysisBackend:\n    module_path, class_name = BACKEND_REGISTRY[name].rsplit(\":\", 1)\n    module = importlib.import_module(module_path)\n    return getattr(module, class_name)()\n```\n\nBackends are imported lazily — if you never use Grok, `grok.py`\n\nis never loaded. Adding a new backend is three steps:\n\nCreate a module that subclasses `ExternalAgentBackend`\n\nImplement `_build_command()`\n\n— return the argv list\n\nAdd one line to `BACKEND_REGISTRY`\n\nHere's the entire Grok backend:\n\n```\nclass GrokBackend(ExternalAgentBackend):\n    name = \"grok\"\n    display_name = \"Grok Build\"\n    cli_command = \"grok\"\n\n    def _build_command(self, prompt, cwd, max_turns):\n        return [\n            \"grok\", \"-p\", prompt,\n            \"--cwd\", cwd,\n            \"--tools\", \"read_file,grep,list_dir\",   # read-only\n            \"--yolo\",\n            \"--output-format\", \"json\",\n            \"--max-turns\", str(max_turns),\n        ]\n```\n\nEverything else — prompt assembly, subprocess management, output parsing, raw artifact saving, and the Stage 2 structurer call — is handled by the `ExternalAgentBackend`\n\nbase class. The concrete backend only knows how to build its CLI command.\n\nIn `config.yaml`\n\n:\n\n```\ncode_analysis:\n  enabled: true\n  backend: \"grok\"        # swap to \"builtin\", \"claude_code\", or \"codex\"\n  max_turns: 15           # how many exploration turns for external agents\n```\n\nThe setup wizard auto-detects which CLIs are installed on your PATH and lets you pick:\n\n```\nAvailable analysis backends:\n  1. builtin     — Built-in (PydanticAI) — lightweight, no extra tools needed\n  2. grok        — Grok Build — headless mode, read-only tool access\n  3. claude_code — Claude Code — deep exploration, 200K context\nChoose a backend [1]:\n```\n\nWhether using the builtin or an external backend, the output is the same validated structure:\n\n``` bash\n$ apppulse analyze 42\n\nAnalyzing review #42 against MyApp Android codebase...\n  Code path: /Users/me/projects/myapp\n  Issue type: bug\n  Backend:   grok\n\n  > Launching Grok Build agent...\n  ... Agent exploring codebase (this may take 30-60s)...\n  OK Raw analysis saved to ~/.apppulse/analyses/raw/MyApp_bug_20250710_091523.md\n  > Structuring analysis into brief...\n  OK Analysis complete\n\n┌──────────────────────────────────────────────────────────────┐\n│  CODE ANALYSIS — Review #42                                   │\n│  Photo upload crashes for >10MB images                        │\n│  Type: bug │ Complexity: medium                               │\n├──────────────────────────────────────────────────────────────┤\n│  ROOT CAUSE                                                   │\n│  ImageUploadService.compress() loads the full bitmap into     │\n│  memory before compression. Images >10MB exceed the heap      │\n│  limit on devices with ≤4GB RAM → OutOfMemoryError.           │\n│                                                               │\n│  AFFECTED FILES                                               │\n│  ImageUploadService.java (lines 45-89) — compress() method    │\n│  ImageUtils.java (lines 12-35) — no input size validation     │\n│                                                               │\n│  PROPOSED CHANGES                                             │\n│  1. Add BitmapFactory.Options.inSampleSize for downsampling   │\n│  2. Add file size validation before compress()                │\n│  3. Use streaming compression instead of in-memory byte array │\n│                                                               │\n│  RISKS                                                        │\n│  • Downsampling reduces image quality for server-side use     │\n│  • Need to test across API levels 26-34                       │\n└──────────────────────────────────────────────────────────────┘\n```\n\nWith `--output file`\n\n, it saves the analysis as markdown with a ready-to-paste prompt section at the bottom — hand it directly to a coding agent to implement the fix.\n\n**Config nesting.** The `code_analysis`\n\nblock goes at the **top level** of your config YAML, not nested under an app entry. I made this mistake myself — the parser reads `code_analysis`\n\nfrom the root, so nesting it under `apps[]`\n\nmeans it silently gets ignored and falls back to defaults.\n\n`prepare_tools`\n\n**is called every turn.** It receives the full message history, so counting tool calls is O(n) over all messages. For 25 turns this is negligible, but if you remove the request limit, it could get slow on very long conversations.\n\n**External agent stdout formats vary.** Claude Code with `--output-format json`\n\nwraps its response in a JSON object; Codex prints plain text. The `ExternalAgentBackend._extract_text()`\n\nmethod handles both, but if you add a new backend, check what it actually writes to stdout.\n\n**Model name prefix matters.** PydanticAI wants `openai-chat:gpt-4o-mini`\n\n(not just `gpt-4o-mini`\n\n). Without the prefix, you get a deprecation warning and potentially the wrong provider.\n\nThis pipeline keeps evolving. Some things on the radar:\n\n**Per-app backend config.** Right now `code_analysis`\n\nis global. If you have multiple apps, you might want Grok for one and builtin for another. The config structure needs to support this.\n\n**Batch analysis.** `apppulse analyze --category bug --severity critical`\n\nto analyze all critical bugs in one run instead of one-by-one.\n\n**Analysis history.** Tracking which analyses led to successful fixes — creating a feedback loop that improves future analysis quality.\n\n**Auto-analyze in pipeline.** Option to run analysis on all critical bugs during `apppulse run`\n\n, so the morning digest includes code-level briefs, not just classification.\n\n**Start with classification, then add analysis.** Getting reviews categorized (bug vs. feature request vs. praise) is the foundation. Code analysis is the layer on top — it's valuable but depends on good classification first.\n\n**Give the agent a map before tools.** The repo map cut tool calls by 60%. An LLM exploring a codebase blind will waste most of its budget on orientation. Front-load structure.\n\n**Funnel, don't limit.** Progressive tool gating works better than a hard request limit. The agent naturally transitions from exploration to deep-dive to output, rather than being cut off mid-investigation.\n\n**Make the backend a strategy, not a hard dependency.** The two-stage pipeline (explore → structure) decouples the exploration engine from the output format. Today it's Grok and Claude Code; tomorrow it could be any agent that can read files and write markdown.\n\n**Save raw artifacts.** The raw markdown from external agents is invaluable for debugging bad analyses. If the structured output looks wrong, the raw artifact tells you whether the problem was Stage 1 (bad exploration) or Stage 2 (bad extraction).\n\n*AppPulse is a personal project I keep iterating on. The code analysis pipeline is the part I'm most excited about — it's the difference between \"you have 3 critical bugs\" and \"here's exactly where they are and what to change.\" If you're building something similar, the pluggable backend pattern and two-stage pipeline might save you some architecture headaches.*", "url": "https://wpnews.pro/news/from-you-have-a-bug-to-here-s-the-root-cause-adding-ai-code-analysis-to-my-app", "canonical_source": "https://dev.to/ashish_mishra_8491c3b9912/from-you-have-a-bug-to-heres-the-root-cause-adding-ai-code-analysis-to-my-app-review-pipeline-51f1", "published_at": "2026-07-21 01:33:50+00:00", "updated_at": "2026-07-21 01:58:54.951653+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "ai-agents", "large-language-models"], "entities": ["AppPulse", "PydanticAI", "OpenAI", "Anthropic", "Ollama", "Google Play", "App Store", "Sentry"], "alternates": {"html": "https://wpnews.pro/news/from-you-have-a-bug-to-here-s-the-root-cause-adding-ai-code-analysis-to-my-app", "markdown": "https://wpnews.pro/news/from-you-have-a-bug-to-here-s-the-root-cause-adding-ai-code-analysis-to-my-app.md", "text": "https://wpnews.pro/news/from-you-have-a-bug-to-here-s-the-root-cause-adding-ai-code-analysis-to-my-app.txt", "jsonld": "https://wpnews.pro/news/from-you-have-a-bug-to-here-s-the-root-cause-adding-ai-code-analysis-to-my-app.jsonld"}}