{"slug": "i-let-an-ai-write-my-tests-for-30-days-coverage-went-38-to-71", "title": "I Let an AI Write My Tests for 30 Days: Coverage Went 38% to 71%", "summary": "A developer reports that using an open-source AI agent called 'the-agent' to generate and maintain tests for 30 days increased code coverage from 38% to 71% without writing a single test by hand. The developer integrated the tool into CI with patch-style generation, which only tests changed files, and highlights pitfalls such as the need for precise descriptions, handling legacy compatibility, and reviewing async cases manually.", "body_md": "Here's the number that surprised me: **30 days, zero tests written by hand, coverage from 38% to 71%.**\n\nI handed test-writing to an open-source AI agent (search **the-agent** on GitHub) and let it generate, run, and maintain my tests from natural-language descriptions. This is the full account — the workflow, the configs, the pitfalls, and the honest trade-offs.\n\nLast month I broke 35 tests by changing one function signature. Fixing them took until lunch. The pain wasn't writing tests — it was *maintaining* them: normal inputs, edge cases, error branches, and the worst part — that false confidence of \"all green\" when critical paths were never covered.\n\nI saw the-agent trending on GitHub (a prompt-based test automation tool that uses AI agents to generate, run, and maintain tests) and decided to run a real 30-day experiment.\n\n```\n# Install\nnpm install -g the-agent\n\n# Init project config\nthe-agent init --project ./my-app --language typescript\n```\n\nGenerated config:\n\n```\n{\n  \"project\": \"my-app\",\n  \"language\": \"typescript\",\n  \"testFramework\": \"vitest\",\n  \"coverageTarget\": 70,\n  \"asyncDetection\": true,\n  \"compatibilityNotes\": \"legacy endpoints keep original format\"\n}\n```\n\nKey flags:\n\n`coverageTarget`\n\n— CI gate threshold`asyncDetection`\n\n— catches missing async waits (critical, see pitfalls)`compatibilityNotes`\n\n— tells the agent about legacy constraintsI asked it to test an order module's `calculateTotal`\n\n:\n\n```\nthe-agent test --describe \"calculateTotal receives product array, computes total, supports coupon discount, 100 off 20\"\n```\n\nIt generated cases covering: normal totals, empty arrays, discount thresholds, coupon stacking, and negative-price exceptions. First run, I was genuinely impressed.\n\nThe trick is **patch-style generation**, not full-suite generation:\n\n```\nname: AI Test Agent\non:\n  pull_request:\n    types: [opened, synchronize]\n\njobs:\n  ai-tests:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n        with:\n          fetch-depth: 0\n      - uses: actions/setup-node@v4\n        with:\n          node-version: 20\n      - run: npm ci\n      - run: |\n          the-agent test --diff origin/main...HEAD \\\n            --config ./the-agent.config.json \\\n            --report ./ai-test-report.json\n      - uses: actions/upload-artifact@v4\n        with:\n          name: ai-test-report\n          path: ai-test-report.json\n```\n\nOnly tests changed files. ~5-8 minutes per PR, and every PR gets an AI-generated coverage patch plus a coverage gate.\n\n**1. Bad description = wrong tests.** I forgot to mention an async confirmation step; it generated all-sync cases that passed falsely. Fix: explicitly state async in the description.\n\n**2. Legacy compatibility.** It writes \"best-practice\" tests that fail against old formats. Fix: declare constraints in `compatibilityNotes`\n\n.\n\n**3. Async gaps.** Timers, callbacks, external calls — occasionally missed timing. `asyncDetection: true`\n\nhelps but doesn't fix everything. Review async cases manually.\n\n**4. It won't think for you.** It guarantees tests *run*, not that your business logic is *right*. Wrong description → confidently wrong tests. My rule: AI generates, I review semantics.\n\nHard coverage gates (block merge below 60%), Python/Go support, and documenting the prompt templates I've collected. AI-assisted testing is becoming mainstream — start now and you'll have a workflow ready when the tooling matures.\n\n*The tool is free and open source. Search \"the-agent\" on GitHub. Save this for when you wire it into your CI.*", "url": "https://wpnews.pro/news/i-let-an-ai-write-my-tests-for-30-days-coverage-went-38-to-71", "canonical_source": "https://dev.to/mage0535/i-let-an-ai-write-my-tests-for-30-days-coverage-went-38-to-71-1ka3", "published_at": "2026-08-02 05:20:18+00:00", "updated_at": "2026-08-02 05:37:17.130571+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "ai-agents", "ai-products"], "entities": ["the-agent", "GitHub", "vitest", "TypeScript", "Node.js", "GitHub Actions"], "alternates": {"html": "https://wpnews.pro/news/i-let-an-ai-write-my-tests-for-30-days-coverage-went-38-to-71", "markdown": "https://wpnews.pro/news/i-let-an-ai-write-my-tests-for-30-days-coverage-went-38-to-71.md", "text": "https://wpnews.pro/news/i-let-an-ai-write-my-tests-for-30-days-coverage-went-38-to-71.txt", "jsonld": "https://wpnews.pro/news/i-let-an-ai-write-my-tests-for-30-days-coverage-went-38-to-71.jsonld"}}