AI generated unit tests, AI coding assistant compa

How to use Claude 3.5 Sonnet and GPT-4o to write unit tests that actually catch bugs

Writing unit tests is the part of coding everyone hates, so we let AI do it. But if you just prompt "write tests for this file," you get "happy path" garbage—tests that pass because they only test what the code already does, not where it breaks.

To get tests that actually save your skin, you need to stop treating the AI as a magic wand and start treating it as a QA engineer who needs specific context. I spent last Thursday fighting with a flaky payment integration; it took three different prompts and a specific RAG-based approach to finally find the race condition that was dropping 2% of my transactions.

Stop prompting for "unit tests" and start prompting for "edge cases" #

If you ask for unit tests, you get a mirror of your code. If you ask for "edge cases that would break this logic," you get a bug hunter.

Here is the workflow that actually works for me in Cursor (using Claude 3.5 Sonnet). Don't just highlight the function. Feed it the type definitions and the existing test suite first.

cat src/types/payment.ts src/services/payment-service.ts > context.txt

Then, use a prompt that forces the AI to think about failure.

The prompt I use:

"Analyze payment-service.ts. Ignore the happy path. List 5 ways this function could fail based on the types in payment.ts (e.g., null pointers, timeout drifts, API 500s). Then, write Vitest tests for those 5 specific failure modes using vi.mock for the gateway."

The difference is night and day. Instead of expect(result).toBe(true), you get tests that simulate a network timeout and check if your retry logic actually triggers.

Which model actually writes better tests? #

I've run a head-to-head comparison between Claude 3.5 Sonnet and GPT-4o on a codebase with about 40k lines of TypeScript. Here is the raw reality of how they perform for AI generated unit tests.

| Metric | Claude 3.5 Sonnet | GPT-4o |

| :--- | :--- | :--- |

| Logic Accuracy | High (catches off-by-one errors) | Medium (often misses edge cases) |

| Boilerplate | Concise | Tends to over-explain/over-comment |

| Mocking | Great at vi.mock / jest.mock | Sometimes hallucinates library methods |

| Context Window | Feels more "aware" of distant files | Occasionally forgets the type defs |

Sonnet is currently the king of coding. It doesn't just write code; it reasons about the state. GPT-4o is faster and cheaper for simple CRUD tests, but for complex business logic, I wouldn't trust it to find a bug without a very tight prompt. If you're choosing AI Models for a team, put Sonnet on the heavy lifting and 4o on the documentation.

Solving the "Context Gap" with RAG retrieval augmented workflows #

The biggest fail point in AI testing is when the AI doesn't know how your internal libraries work. It'll suggest a method like .fetchUser() that doesn't exist because it's guessing based on common patterns.

This is where RAG (Retrieval-Augmented Generation) comes in. You don't need to build a complex vector database from scratch. If you're using a tool like Cursor or Windsurf, they are doing RAG under the hood when you use @Codebase.

But the "automatic" RAG often misses the mark. To fix this, I manually "prime" the context. When I hit a wall with a hallucinated method, I do this:

1. Open the actual source file of the dependency.

2. Keep it open in a tab (most AI assistants prioritize open tabs).

3. Explicitly mention the file: "Use the method signatures found in api-client.ts to mock the response."

If you're building your own AI agent for testing, don't just dump the whole folder into the prompt. Use a hybrid approach: retrieve the class definition via semantic search, but hard-code the most used utility functions into the system prompt. It cuts hallucinations by about 30% in my experience.

Dealing with the "Passes but is Useless" trap #

There is a specific type of failure where AI generates a test that passes, but it's testing the mock, not the code.

Example:

AI writes a mock that returns true, then writes a test that expects true.

Result: Green checkmark.

Reality: Zero coverage of actual logic.

To kill this, I implement a "Mutation Test" prompt. Once the AI generates the test, I ask:

"If I change line 42 from > to >=, will this test fail? If not, rewrite the test to be more sensitive to boundary conditions."

This forces the AI to evaluate the effectiveness of the test, not just the syntax. It's tedious, but it's the only way to ensure your test suite isn't just a vanity metric.

Putting it into practice #

If you're just starting, don't try to automate your whole suite. Start with one problematic file.

1. Use a tool that supports deep codebase indexing.

2. Feed it the types, then the logic.

3. Demand edge cases over happy paths.

4. Verify the mocks aren't just echoing the output.

For those who want to see how others are structuring their prompts for different frameworks, checking out the Resources section of a dev community can save you hours of trial and error. I found a specific pattern for testing React hooks there that cut my boilerplate code by half.

The wild part is that the AI is often better at finding the bug than it is at fixing it. Use it as a diagnostic tool first, and a code generator second.

Next How Automatic Key Exchange Slashed TLS 1.3 HRRs from 52% to 3% →