cd /news/ai-tools/why-ai-built-apps-feel-fast-in-testi… Β· home β€Ί topics β€Ί ai-tools β€Ί article
[ARTICLE Β· art-58681] src=dev.to β†— pub= topic=ai-tools verified=true sentiment=↓ negative

Why AI-Built Apps Feel Fast in Testing and Break in Production

A developer built a client reporting tool with Bolt.new that worked well initially but failed under concurrent load at 80 users. The AI-generated code was correct but not optimized for production, leading to issues like missing indexes, under-configured connection pools, N+1 queries, and cold starts. The developer spent a week profiling and fixing these problems, which were structurally invisible during solo testing.

read7 min views1 publishedJul 14, 2026

I shipped a client reporting tool built with Bolt.new. Three weeks from first prompt to first paying user. Everything worked.

At eleven paying customers, everything still worked. At forty-three, it started getting slow. At eighty simultaneous users, it started timing out. At the exact moment a potential enterprise customer was watching a live demo with six of their colleagues, it went fully unresponsive.

I closed the laptop. Said I would follow up.

Then I spent a week figuring out what I had actually built β€” and more importantly, what the AI tools had quietly assumed about scale.

When I went back into the codebase after that demo, I was not looking for bugs. The app was not buggy. It did the right thing. It returned the right data. The logic was correct.

What it was not was efficient under concurrent load.

And this is the distinction that nobody in the vibe coding space talks about clearly enough:

Correct and performant are the same thing at ten users. They are very different things at five hundred.

AI coding tools write code that is correct. They are not optimizing for what happens when thirty users hit the same endpoint simultaneously, all triggering database queries, all competing for the same connection pool. That failure mode is outside the context of any single prompt. The model generates working code. It does not generate production-optimized code, because production optimization requires knowing what load looks like β€” and that knowledge does not exist at prompt time.

After a week of profiling and reading the codebase, I found four distinct issues. None of them were exotic. All of them were completely predictable in hindsight.

Missing indexes. The AI had generated queries that filtered by agency ID, date range, and status. Correct queries. But there were no indexes on any of those columns. At a few hundred rows, the query planner just scanned the whole table β€” fast enough to be invisible. At eighty thousand rows, those same queries were taking four to eight seconds each. Adding indexes to the three most-queried columns fixed roughly 70% of the total latency.

Under-configured connection pool. The default connection pool settings in the AI-generated backend were calibrated for a development environment. Five simultaneous connections. When eighty users were on the platform, requests started queuing. Then timing out. Then users refreshed β€” generating more requests, longer queues, more timeouts. The intermittent errors I had been chalking up to transient network issues were actually systematic connection exhaustion happening every time usage spiked.

N+1 queries on the dashboard. The main dashboard loaded a list of client projects. For each project, it displayed the associated client name. The AI had written this as two database operations: one query to get all projects, then one query per project to get the client name. Fifty projects on the dashboard meant fifty-one database queries on every page load. At low user counts this was invisible. At scale it was a disaster β€” a page that should have taken one database round trip was taking fifty-one.

Cold starts on idle infrastructure. The deployment environment scaled to zero during off-hours. The first user to log in each morning triggered a cold start β€” the server had to spin up from scratch before serving the request. Two to six seconds of for the first user. Fine for a solo founder testing their own tool. A terrible experience for an agency employee who just wanted to pull a report before their 9am client call.

I want to be specific about this because I spent a while blaming myself for not catching these issues earlier.

These four problems are structurally invisible during solo testing. Not hard to find β€” structurally invisible. They do not exist at small scale. They emerge only when multiple users interact with the system simultaneously, competing for shared resources.

The missing index is imperceptibly fast at a thousand rows. The connection pool issue never triggers with five users. The N+1 query runs in milliseconds when there are ten rows. Cold starts happen once a day and feel like a fluke.

The only way to find them before your users do is to simulate load deliberately β€” to run the app with multiple concurrent sessions hitting the core user flows simultaneously, and watch what the database and server do.

AI tools cannot do this for you, because they generate code without a model of what production load looks like. The model builds the feature. It does not build the production version of the feature. That distinction only becomes meaningful at a scale the tool never tested against.

The work was not complicated. It was methodical.

First, I enabled slow query logging on the database and ran a realistic multi-user simulation β€” twenty concurrent sessions running through the main dashboard and report generation flows. The slow query log surfaced the three worst-offending queries immediately.

Second, I looked at those queries, found the missing indexes, added them, and re-ran the simulation. The page load times dropped from four to eight seconds to under 300 milliseconds on the same queries.

Third, I fixed the N+1 on the dashboard β€” rewrote the two-step data fetch as a single joined query. Fifty-one database round trips became one.

Fourth, I reconfigured the connection pool to match the concurrency profile I was actually seeing. Timeouts stopped.

Fifth, I added a keep-warm ping to prevent cold starts during business hours.

The whole thing took about a week of focused work. None of the changes touched the core product logic. None of them required rewriting anything. They were all targeted interventions in specific, identifiable bottlenecks.

There is a version of your app that works, and a version of your app that holds. AI tools are extraordinary at getting you to the version that works. The version that holds requires someone to look at the system with a production lens β€” to ask not "does this return the right data" but "what does this do when a hundred people ask for this data at the same time."

That is not a criticism of the tools. They did exactly what they are designed to do. They got me to a working product in three weeks instead of three months. That is a genuine, real, significant compression of time and effort.

But the production gap is real. And pretending it is not β€” shipping the AI-generated v1 to users at scale without profiling it, without simulating load, without checking for the patterns that emerge under concurrency β€” is the thing that caused the demo call incident, not the tools.

The founders I have talked to who handle this transition best are the ones who treat the AI-generated codebase as a starting point for a production audit rather than a finished product. Read the queries. Check the indexes. Simulate load before launch. Find the N+1 patterns before users do.

For the apps where the scaling issues are deep enough that they are hard to find quickly β€” where the codebase has grown large, the issues are layered, and the audit itself requires knowing what to look for β€” having engineers who have read a lot of these AI-generated exports specifically is what speeds up the process. The team at IT Path Solutions built an entire practice around this because the pattern keeps repeating: correct app, missing production optimizations, scaling crisis that did not have to happen. Profiling, targeted fixes, no rewrite required. I do not wait for users to find the performance issues anymore.

Before any significant launch, I run a basic load simulation against the core user flows. I check the database for missing indexes on any column being filtered or sorted. I look at the ORM-generated queries for N+1 patterns. I check the connection pool configuration against the concurrency I expect.

None of this is complicated. All of it is things I should have done before that demo call.

The vibe coding tools gave me the fastest v1 I have ever shipped. Learning how to validate that v1 for production before it meets real users β€” that part was on me.

Have you hit a scaling wall in an AI-built app? What was the root cause? Drop it in the comments β€” curious what patterns others are seeing.

── more in #ai-tools 4 stories Β· sorted by recency
── more on @bolt.new 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain β€” perfect for shipping the agent you just read about.

$git push zahid main
β†’ Live at https://your-agent.zahid.host βœ“
Get free account β†’ Pricing
from €0/mo Β· no card required
LIVE [news/why-ai-built-apps-fe…] indexed:0 read:7min 2026-07-14 Β· β€”