{"slug": "hiring-and-firing-a-dba", "title": "Hiring and Firing a DBA", "summary": "Buttondown, a newsletter platform, ran a 60-day experiment using GPT 5.3 Codex High to automatically generate database performance improvements, which cut database reads by 25%, but the company ended the experiment because the LLM ran out of easy targets and overclaimed its improvements. The experiment's approval rate dropped from 75% in the first 30 days to 30% in the latter 30, and three rejected PRs would have caused incidents if merged.", "body_md": "# Hiring and firing a DBA\n\n## Hiring our DBA\n\nFor the past sixty days, Buttondown I take liberties here and throughout the essay: this was all Matias' doing. He [also wrote a great piece on our migration to Planetscale](https://buttondown.com/blog/how-we-migrated-our-database-to-planetscale), and also cites Planetscale's recommendation for this experiment. ran a scheduled LLM invocation once a day. Every morning it would look at the previous few days of database load in [PlanetScale](https://planetscale.com), and — if, and only if, it was sufficiently confident — it would open a pull request containing exactly one performance improvement to the seam between our app and our database.\n\nI think of this as a longitudinal study, and a fairly cheap one: LLMs are extremely well suited to performance optimizations, precisely because such optimizations can be evaluated and then trivially discarded. We used GPT 5.3 Codex High for this, but I suspect our outcomes would have been broadly similar with whatever model we reached for.\n\nTwo months down, and here's where we're at:\n\n1. We decreased our overall database reads by 25% for no cost or effort beyond that of reviewing the PRs.\n2. We are stopping the experiment.\n\nLet me frame the rest of the post with the most important takeaway for a prospective reader: this was an undoubtedly positive-value experiment, and I recommend everyone try it if they're able. Even setting the epistemology aside, the performance benefits we banked were unambiguously worth the small amount of time I'd spend reviewing PRs every morning. The only reason I'm not dwelling on the upside is that I think the flaws are more instructive than the wins.\n\nSo, with that out of the way —\n\n## Firing our DBA\n\n25% improvement! That's great! Why stop the party?\n\n**The single biggest reason is that the LLM ran out of easy targets.** In the first thirty days, I approved 75% of incoming PRs. In the latter thirty, that number dropped to 30%.\n\nThe theoretical underpinning of the whole enterprise is that there's a sweet spot: the risk of a given PR has to round down to zero, while the value of the performance benefit has to stay high enough to outweigh the attention and energy required to review it. This will be covered in a post to come, but a note I'd have here for the \"get rid of code review\" maximalists is that at three of the PRs I rejected would have caused an incident if merged. Without looking too hard at the Insights dashboard, it's clear the LLM ran out of obvious wins and it started getting desperate. Most of the PRs it submitted toward the end were bad code that made choices I didn't like — adding branching logic to defer work under some very esoteric circumstance, say, which would have a real positive performance impact and an equally real, and larger, negative technical-debt one.\n\n**The second reason is that the LLM consistently overclaimed** — both the scale of the improvement it was making and its confidence in being able to make any change whatsoever.\n\nI led with the exciting headline: it cut 25% of our reads. Which is great! But that impressive feat is itself roughly 25% of what it *claimed* it was going to do. Yes, the math there is not a typo. Largely, I think, this is because we did not — for obvious reasons — give it access to the actual production dataset, only the metadata about it. So it was calibrating a lot of its estimates against synthetic datasets it would conjure up itself, and synthetic data is a flattering mirror.\n\nThe combination of the two meant this experiment had crossed from positive value into negative-value territory.\n\n## What about upskilling?\n\nI'm entirely amenable to the counterargument that I should treat this as a harness-engineering problem instead: give the model better instructions, or tooling to approximate the prod dataset, or coaching on which sacrifices to refuse. And those responses are correct, in a certain sense: they probably would have resulted in a better signal-to-noise ratio. But they still amount to me spending unjustifiable time on this experiment. It was valuable to me *because* it was free real estate, so to speak — not because I have a deep and systemic database problem. (Once again: thank you to PlanetScale for making that no longer the case.)\n\nZoom out further and you arrive at the worry that haunts everything shaped like this: there really is only so much low-hanging fruit. Every time a frontier lab ships a new model your proverbial ladder gets a little taller, but at some point you have to move on from your current copse of trees. Which is not to say ladders aren't useful, only that they're not jetpacks.", "url": "https://wpnews.pro/news/hiring-and-firing-a-dba", "canonical_source": "https://www.jmduke.com/posts/hiring-and-firing-a-dba.html", "published_at": "2026-09-08 06:15:00+00:00", "updated_at": "2026-09-08 06:31:36.528933+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "ai-agents"], "entities": ["Buttondown", "Matias", "PlanetScale", "GPT 5.3 Codex High"], "alternates": {"html": "https://wpnews.pro/news/hiring-and-firing-a-dba", "markdown": "https://wpnews.pro/news/hiring-and-firing-a-dba.md", "text": "https://wpnews.pro/news/hiring-and-firing-a-dba.txt", "jsonld": "https://wpnews.pro/news/hiring-and-firing-a-dba.jsonld"}}