I Let an AI Write My Tests for 30 Days: Coverage Went 38% to 71%

Here's the number that surprised me: 30 days, zero tests written by hand, coverage from 38% to 71%.

I handed test-writing to an open-source AI agent (search the-agent on GitHub) and let it generate, run, and maintain my tests from natural-language descriptions. This is the full account — the workflow, the configs, the pitfalls, and the honest trade-offs.

Last month I broke 35 tests by changing one function signature. Fixing them took until lunch. The pain wasn't writing tests — it was maintaining them: normal inputs, edge cases, error branches, and the worst part — that false confidence of "all green" when critical paths were never covered.

I saw the-agent trending on GitHub (a prompt-based test automation tool that uses AI agents to generate, run, and maintain tests) and decided to run a real 30-day experiment.

npm install -g the-agent

the-agent init --project ./my-app --language typescript

Generated config:

{
  "project": "my-app",
  "language": "typescript",
  "testFramework": "vitest",
  "coverageTarget": 70,
  "asyncDetection": true,
  "compatibilityNotes": "legacy endpoints keep original format"
}

Key flags:

coverageTarget

— CI gate thresholdasyncDetection

— catches missing async waits (critical, see pitfalls)compatibilityNotes

— tells the agent about legacy constraintsI asked it to test an order module's calculateTotal

:

the-agent test --describe "calculateTotal receives product array, computes total, supports coupon discount, 100 off 20"

It generated cases covering: normal totals, empty arrays, discount thresholds, coupon stacking, and negative-price exceptions. First run, I was genuinely impressed.

The trick is patch-style generation, not full-suite generation:

name: AI Test Agent
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  ai-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: |
          the-agent test --diff origin/main...HEAD \
            --config ./the-agent.config.json \
            --report ./ai-test-report.json
      - uses: actions/upload-artifact@v4
        with:
          name: ai-test-report
          path: ai-test-report.json

Only tests changed files. ~5-8 minutes per PR, and every PR gets an AI-generated coverage patch plus a coverage gate.

1. Bad description = wrong tests. I forgot to mention an async confirmation step; it generated all-sync cases that passed falsely. Fix: explicitly state async in the description.

2. Legacy compatibility. It writes "best-practice" tests that fail against old formats. Fix: declare constraints in compatibilityNotes

.

3. Async gaps. Timers, callbacks, external calls — occasionally missed timing. asyncDetection: true

helps but doesn't fix everything. Review async cases manually.

4. It won't think for you. It guarantees tests run, not that your business logic is right. Wrong description → confidently wrong tests. My rule: AI generates, I review semantics.

Hard coverage gates (block merge below 60%), Python/Go support, and documenting the prompt templates I've collected. AI-assisted testing is becoming mainstream — start now and you'll have a workflow ready when the tooling matures.

The tool is free and open source. Search "the-agent" on GitHub. Save this for when you wire it into your CI.