The hype around AI coding agents has reached a point where "I use Cursor" or "I use Claude Code" is becoming a default answer in developer conversations. But the gap between what people claim works and what actually works in daily workflows is still wide. I spent 30 days using multiple agents on real projects, not toy examples, and the results were more nuanced than either the evangelists or the skeptics suggest.
I ran three agents across different tasks over a month: Claude Code (Anthropic), GitHub Copilot CLI, and Cursor in agent mode. Same codebase, same problems, same evaluation criteria. No cherry-picking wins.
The projects were not demos. A small SaaS API with auth, webhooks, and a React admin panel. A data pipeline with Python and SQL. A legacy Node.js service that needed refactoring.
What I measured:
Agents excel at generating CRUD endpoints, database migrations, and API route scaffolding. The time savings here are real, not marginal. A set of 12 REST endpoints that would take me 45 minutes of copy-paste and boilerplate took about 8 minutes with an agent.
@router.post("/register", response_model=UserResponse)
async def register(user_in: UserCreate, db: AsyncSession = Depends(get_db)):
existing = await db.execute(select(User).where(User.email == user_in.email))
if existing.scalar_one_or_none():
raise HTTPException(status_code=409, detail="Email already registered")
user = User(**user_in.model_dump())
user.hash_password()
db.add(user)
await db.commit()
return user
The code was correct on the first pass. That is unusual for AI-generated code and worth noting.
When I needed to understand a legacy codebase I hadn't touched in months, agents were surprisingly good at tracing call chains and explaining architecture. "Show me how auth tokens flow through this service" produced a useful diagram in seconds.
Writing tests for existing code is tedious. Agents handled this well for straightforward unit tests but struggled with integration tests that required understanding of external service contracts.
When I asked agents to implement a multi-step payment reconciliation flow with edge cases for failed webhooks, partial refunds, and idempotency, the output was wrong about 60% of the time on the first pass. The code looked plausible but missed subtle state transitions.
On larger codebases, agents started losing track of imports and type definitions. Cursor handled this better than the CLI-based tools, but even it would occasionally suggest methods that didn't exist on a model it had seen 20 files ago.
Agents are not good at debugging issues they cannot reproduce. When a bug only manifests under specific data conditions in production, the agent's suggestions were generic at best and misleading at worst.
| Task Type | Agent Speedup | First-Pass Quality | Merge-Ready |
|---|---|---|---|
| Boilerplate | 4x faster | 90% | Yes |
| Refactoring | 2x faster | 75% | With review |
| New feature | 1.5x faster | 60% | No |
| Bug fix | 1x (slower) | 40% | No |
These are rough numbers from my usage, not a controlled benchmark. Your mileage will vary based on codebase complexity and how well you write prompts.
The biggest change was not speed. It was context switching. Instead of holding the entire problem in my head while typing, I could describe the problem, review the agent's output, and iterate. The cognitive load shifted from "write every line" to "review and direct."
That shift is real but comes with a cost: you need strong enough mental models to review the agent's work. If you don't understand the code the agent produces, you are not using an agent, you are delegating to a black box.
If you are doing repetitive work on familiar codebases, agents will save you time today. If you are building novel systems or debugging tricky issues, treat agents as a junior developer that needs supervision, not a senior engineer that works autonomously.
The tools are improving fast, but the fundamental constraint remains: agents are only as good as the context you give them and the review you do after.
AI coding agents are not a replacement for developers. They are a productivity multiplier for specific tasks, and a liability for others. The developers who will get the most out of them are the ones who understand the codebase well enough to review the agent's output critically.
What tasks have you found agents actually helpful for, and where did they disappoint you? I am curious whether my experience matches what others are seeing in their workflows.
Tags: ai, programming, developer-tools, claude-code
What's one task where an AI coding agent genuinely surprised you with the quality of its output? I am looking for specific examples, not general impressions.