I read a post on Lobste.rs last week about dreading our LLM-written incident report future, and it hit a massive nerve. We've reached a point where the moment production stops burning, our next instinct's to automate the thinking.
I'm tired of reading beautifully formatted incident reviews that read like they're written by a consultant who's never seen a terminal. They've all the right sections, the timelines are perfectly clean, and they explain absolutely nothing about why we actually broke.
This connects to something I was chewing on last week when I wrote about the cultural bankruptcy of tracking developer activity through screen recordings. If you treat engineering like a factory where every input must be logged and optimized, you end up destroying professional trust. Writing incident reports with AI's the same disease, just looking from a different angle. It treats the post-mortem as a chore to be automated, rather than the literal mechanism of how we learn as an organization.
Let me tell you how this plays out in reality. Last month, we'd a really nasty incident in the Incident Response channel on Microsoft Teams. It started around 4 PM on a Thursday. Our product detail pages started throwing server errors. This was a nightmare because we were right in the middle of a high-profile marketing campaign.
It wasn't a sudden spike in traffic. For the first hour, everyone thought it's a bad deployment from a catalog team. We rolled back their latest release, but the errors kept climbing. The dashboards showed database CPU was at 98%, but our database logs weren't showing any long-running queries, which was weird.
People were arguing in the Microsoft Teams channel. We almost ran three conflicting recovery scripts at the same time (actually, that's not quite right, we actually had three people trying to run different diagnostic scripts at once because nobody knew who was leading the recovery).
It turned out that a change to our caching logic on the cloud platform had caused a massive cache stampede. Every single product page hit was bypassing the cache and hitting the database directly. It took four hours of stressful, chaotic coordination to write a quick script to pre-warm the cache, rate-limit incoming catalog requests, and gradually ramp traffic back up.
A few days later, we needed to do the post-mortem. Our sprint planning was starting, we were behind on the migration, and the tech lead was drowning in meetings. They did what any busy engineer does these days. They grabbed the Microsoft Teams transcript, dumped a handful of logs from our log viewer, and asked a model to write the incident review.
The document it produced was gorgeous. It'd a clean timeline, a section on the root cause, and five neat action items. I saw the document, knew there's a smell to it, but I was tired and said nothing when I approved it.
That's a mistake.
The problem with an AI-generated post-mortem's that it optimizes for legibility, not truth. An LLM looks at a chaotic chat log and tries to find a clean, logical narrative. But real incidents are messy. They're full of human confusion, false starts, and bad assumptions.
The LLM didn't capture that the only reason we figured out it's a cache stampede's because a senior engineer on the checkout team happened to remember a similar issue from three years ago. It didn't capture that our visualization tool for the cache was showing green because it was measuring the wrong metric (which, to be fair, was running on an older cluster setup). It didn't capture the complete lack of coordination on who was running scripts.
When you feed an incident history into a model, you're asking it to sanitize the friction. But the friction's where the learning happens. If your post-mortem doesn't capture the fact that your team spent two hours chasing the wrong lead because the alerting tool was lying to them, you'll never fix the alerting tool. You'll just keep deploying clean documents while production keeps breaking.
Look, we'll stop treating post-mortems as a compliance exercise. They aren't homework we need to get out of the way so we can go back to shipping features. They're the actual history of our systems. If you automate that memory, you're outsourcing your team's understanding of their own architecture.
The next time we've a cache issue, the engineer who remembers the old outage might be on holiday. The runbook still won't be updated because the LLM didn't realize the runbook was wrong. It just saw a message saying someone's working on it and assumed it was resolved.
So I'm making a decision. From today, I'm rejecting any post-mortem that feels too clean, and I'm personally sitting in on the review meetings for any major outage. I'm going to require that the timeline includes the things we got wrong during the incident, written by the hands of the people who got them wrong. I don't know if this'll slow down our delivery metrics this quarter, and the teams might hate me for adding more friction when they're already busy, but we can't keep pretending we're learning when we're just outsourcing our thinking.