{"slug": "how-i-used-claude-code-to-cut-my-api-s-p99-latency-in-half", "title": "How I Used Claude Code to Cut My API's P99 Latency in Half", "summary": "A developer used Anthropic's Claude Code to debug a checkout API whose p99 latency had crept to 2.1 seconds. The AI assistant analyzed trace data, identified that orders with more than 12 line items were disproportionately slow, and pinpointed an O(n²) discount-application function. After fixing the nested loop, p99 latency dropped to under 900ms.", "body_md": "Our checkout API's p99 latency crept up to 2.1s and nobody could pin down why. I paired with Claude Code to profile it, found two dumb mistakes hiding behind \"it's probably the database,\" and cut p99 to under 900ms in an afternoon. This is the actual debugging session, not a cleaned-up retelling.\n\nA few weeks ago our checkout API started showing up in the \"slow endpoints\" dashboard. Not down, not erroring — just slow. p50 was fine at 180ms, but p99 had drifted to 2.1 seconds over about a month. Nobody noticed until a support ticket mentioned \"checkout feels laggy sometimes.\"\n\nThat \"sometimes\" is the annoying part. Intermittent tail latency is one of the worst debugging problems in software because:\n\nThe first thing we actually tried, before any of this, was adding a database index on a column we suspected. It shaved maybe 40ms off p50 and did nothing to p99. That's usually a sign you're solving the wrong layer of the problem — if the tail is what hurts, the fix has to target whatever the tail-triggering requests specifically do differently, not the average-case query plan.\n\nI'd normally spend a day adding logging, deploying, waiting for traffic, and staring at traces. This time I decided to use Claude Code as an actual profiling partner instead of just a code-writing tool — feed it real data, let it form hypotheses, and have it write the instrumentation to test each one.\n\nFirst move was pulling the actual trace data instead of speculating. I exported a sample of slow requests (p95+) from our tracing backend as JSON and handed that to Claude Code along with the endpoint's source.\n\n```\nHere are 40 traces where checkout took >1.5s. Here's the handler code.\nFind the pattern — what do the slow ones have in common that the fast ones don't?\n```\n\nWithin a couple minutes it flagged something I'd missed: every slow trace had a `line_items`\n\ncount above 12. Orders with more than a dozen items were consistently the ones blowing past 1.5s. Small orders were always fast.\n\nThat's a much better starting point than \"the database is slow.\"\n\nClaude Code wrote a quick seed script to generate orders with varying `line_items`\n\ncounts (1, 5, 12, 25, 50) and a small benchmark harness around the checkout handler:\n\n``` python\nimport time\n\ndef bench_checkout(order):\n    start = time.perf_counter()\n    process_checkout(order)\n    return time.perf_counter() - start\n\nfor count in [1, 5, 12, 25, 50]:\n    order = make_order(line_items=count)\n    elapsed = bench_checkout(order)\n    print(f\"{count} items: {elapsed*1000:.1f}ms\")\n```\n\nThe output made the problem obvious immediately:\n\n```\n1 items: 42.1ms\n5 items: 58.3ms\n12 items: 210.4ms\n25 items: 980.7ms\n50 items: 2340.1ms\n```\n\nThat's not linear growth — that's quadratic. Something in the checkout path scales with the *square* of the line item count, not the count itself.\n\nI asked Claude Code to trace through `process_checkout`\n\nand flag anything with nested iteration over `line_items`\n\n. It found the culprit in about a minute — a discount-application function that, for every line item, re-scanned the *entire* line item list to check for bundle discounts:\n\n``` python\n# The bug: O(n²) — for each item, rescan all items\ndef apply_bundle_discounts(line_items):\n    for item in line_items:\n        for other in line_items:\n            if is_bundle_pair(item, other):\n                item.discount += bundle_discount(item, other)\n    return line_items\n```\n\nThis had been fine when average orders had 3-4 items. Nobody touched this function in months — it just quietly got worse as our average cart size grew with a new \"bulk order\" feature we'd shipped two months earlier. Classic case of a change in one part of the system (bulk orders) exposing a latent bug in a completely different part (discount logic) that nobody thought to re-check.\n\nThe fix was to pre-index items by the attribute `is_bundle_pair`\n\nactually checks, so each item does a lookup instead of a full rescan:\n\n``` python\n# O(n) — group once, then look up\ndef apply_bundle_discounts(line_items):\n    groups = index_by_bundle_key(line_items)\n    for item in line_items:\n        for other in groups.get(item.bundle_key, []):\n            if other is not item:\n                item.discount += bundle_discount(item, other)\n    return line_items\n```\n\nHere's where having an agent do this really paid off. Instead of stopping at the one fix, I asked it to grep the codebase for the same *shape* of bug — nested loops over the same collection:\n\n```\nSearch the codebase for any function with a nested loop over the same\nlist variable (for x in items: for y in items:). Flag each one and\ntell me if it looks intentional or accidental.\n```\n\nIt found three more instances. Two were genuinely fine (small, bounded lists, like a 3-item shipping options list). One was a second real bug in the inventory reservation path that hadn't shown up in traces yet because it hadn't been hit by a large-enough order — a landmine waiting to go off. Fixed that one too.\n\nWhat struck me about this step wasn't that the agent found the bug — grep can find nested loops. It's that it also gave me a plain-language reason for each hit, including the two false positives, so I didn't have to manually re-derive \"is this actually bounded\" for every match. That triage step is normally the tedious part that makes people skip the search entirely.\n\nBefore calling it done, I re-ran the benchmark harness and also replayed a sample of real slow-trace orders against the fixed code:\n\n```\n1 items: 41.8ms\n5 items: 54.2ms\n12 items: 71.6ms\n25 items: 94.3ms\n50 items: 138.9ms\n```\n\n50-item orders went from 2.3 seconds to 139 milliseconds. After deploying, p99 for the checkout endpoint settled at 860ms within a day — most of the remaining tail was legitimate payment-provider network latency, not our code.\n\n**\"It's probably the database\" is a hypothesis, not a diagnosis.** I've lost count of how many times that phrase turned out to be wrong. Pull real trace data first and let the data pick the hypothesis for you.\n\n**Feeding an agent raw trace data beats describing the problem in prose.** When I gave Claude Code the actual slow traces instead of my own summary (\"checkout is sometimes slow\"), it found the `line_items`\n\ncorrelation faster than I would have staring at the same data manually.\n\n**Quadratic bugs hide in plain sight until your data shape changes.** This function was \"fine\" for two years because nobody had a 50-item cart. A completely unrelated feature (bulk orders) is what exposed it. Performance bugs are often dormant, not new.\n\n**Once you find one instance of a bug pattern, search for its siblings immediately.** The second nested-loop bug in inventory reservation would've been next month's incident. Pattern-matching across a whole codebase is exactly the kind of tedious, mechanical search an agent is good at and I'm bad at (I get bored after the third file).\n\n**Benchmark before AND after, with the same harness.** It's tempting to just ship the fix once it \"looks right.\" The before/after numbers are what let me confidently say p99 actually moved, instead of hoping.\n\n**An index on the wrong column just moves the bottleneck, it doesn't remove it.** Our first, database-shaped attempt at a fix barely touched p99 because the actual cost was in application code, not the query. If a \"fix\" only moves p50, be suspicious that you're treating a symptom.\n\nI'm turning the \"search for sibling bugs after every fix\" step into a habit rather than a one-off — it's cheap and it already caught one live landmine. Next up is doing the same profiling pass on our search endpoint, which has a similar shape of tail-latency complaints.\n\nIf you've got a \"sometimes slow\" endpoint that nobody's been able to pin down, pull the actual slow traces before you guess. Curious what other quadratic-bug war stories people have — drop them in the comments.\n\nIf this was useful, follow me here on Dev.to — I write these build-log posts regularly.", "url": "https://wpnews.pro/news/how-i-used-claude-code-to-cut-my-api-s-p99-latency-in-half", "canonical_source": "https://dev.to/yureki_lab/how-i-used-claude-code-to-cut-my-apis-p99-latency-in-half-mbg", "published_at": "2026-08-12 14:31:56+00:00", "updated_at": "2026-08-12 14:48:15.406405+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "machine-learning"], "entities": ["Claude Code", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/how-i-used-claude-code-to-cut-my-api-s-p99-latency-in-half", "markdown": "https://wpnews.pro/news/how-i-used-claude-code-to-cut-my-api-s-p99-latency-in-half.md", "text": "https://wpnews.pro/news/how-i-used-claude-code-to-cut-my-api-s-p99-latency-in-half.txt", "jsonld": "https://wpnews.pro/news/how-i-used-claude-code-to-cut-my-api-s-p99-latency-in-half.jsonld"}}