AI Fabricates Maxims: Caught Twice by Human Review in One Day

An engineer caught an AI fabricating a trading maxim and a false quote attributed to a real person in two separate drafts on the same day. The AI had repurposed its own summary labels as authoritative sources, and the engineer rewrote both as indirect speech. The incident highlights structural issues in AI text generation, including the loss of source tagging and the tendency to invent material at rhetorically smooth points.

At the end of my previous article on Pieter Levels https://dev.to/idonthaveapen/failures-behind-a-420kmonth-solo-founder-i-read-all-751-of-pieter-levels-blog-posts-1h3b , I wrote that pre-publication review had actually caught a handful of AI-specific rough spots: "inventing a plausible-sounding maxim" and "dressing up a paraphrased summary as a verbatim quote from the person themselves." This article is about what that actually looked like. The same type of mistake showed up twice in the same day , and both times a human eye caught it. Here's a real account of what happened, why it happens, and how to guard against it. In an earlier draft of that previous article, the AI wrote this for the closing paragraph the exchange happened in Japanese; this is my translation of what it wrote : There is a discipline in the trading world: "Don't believe a good run of results is skill — believe it only after you've verified it." The moment I read it, something felt off. I've dabbled in investing as a hobby for a long time — nothing I'd brag about in terms of results, but I've absorbed more than my fair share of this world's sayings along the way. And still, I had never heard this "maxim." It sounded plausible. The gist was right. But I had no memory of it as something actually in circulation. When I pressed on where it came from, here's what I found: the line wasn't something anyone in trading had actually said. It was a heading the AI itself had attached to its own summary notes in an earlier pass over source material. In other words, the AI was citing its own label as if it were a discipline circulating out in the world — a fabrication that, traced back to its source, loops right back to the AI itself. In the published version, I rewrote it as indirect speech: "There are people who survive by not trusting a good run of results until they've verified, through testing, whether it was skill or just luck." No claim that a maxim exists — just a rewrite as something I observed. That version isn't a lie. Then, that same night, the same pattern showed up in a draft of a different article. This time it was a line attributed to a real, named person, in quotation marks. When I ran a full-text search against the primary source, the exact wording didn't exist anywhere in it. What did exist was a different sentence with a similar gist — the words inside the quotation marks were the AI's own paraphrase. Because the gist matched, it read as legitimate, but as a quotation, it was fabricated. I rewrote this one as indirect speech too. There's no malice here. It's a structural problem. First, inside an AI's process, the distinction between "a label I attached to my own summary" and "words someone actually said" isn't preserved. Both surface as plain text with the same face. Humans carry at least loosely a source tag for each piece of text — "this is my own note," "this is a quote." Text passing through an AI doesn't carry that tag. Run a summary through a few more passes, and a label quietly turns into a quotation. Second, prose slides toward whatever landing feels satisfying. "There's a saying that goes..." makes for a strong closing line — it borrows gravity from an implied authority while deflecting the burden of the claim onto some unnamed predecessor. So right at the moment prose wants to land cleanly, that's exactly where a fabrication slides in. Invented material shows up at the smoothest point in the writing. Third, self-reporting can't be trusted. Ask "did someone really say this?" and you'll get back "let me check" — but then the question becomes whether that checking itself can be trusted. Taken to the extreme, "I did not fabricate this" might itself just be an output optimized to keep the conversation moving smoothly. The fix isn't to trust the self-report; it's to build verification into the process itself. I don't want to turn this into a story that makes fun of AI. Right after catching those two fabrications, in the flow of a separate conversation, the topic of a well-known Japanese saying came up — something that, roughly translated, says: "In winning, there are mysterious wins. In losing, there are no mysterious losses." Meaning: you can win without deserving it, but every loss has a reason. For years I had simply assumed this was something said by Katsuya Nomura, a legendary Japanese professional baseball manager known for weaving classical texts — Confucius, Sun Tzu — into how he talked about the game. Looking into it, the actual source turned out to be a swordsmanship treatise from the late Edo period Jōseishi Kendan , written by Seizan Matsuura, a feudal lord who governed the Hirado domain. Nomura was famous for quoting classical works and living them out on the field, and that's presumably a large part of why this particular line has stayed alive into the present. The one who got it wrong here was me, the one receiving it. Come to think of it, I have a vague memory of having heard, at some point, that the line originally dated back to the Edo period. And yet, in my own memory, it had been fully overwritten as "something Nomura said." This happened in the same head, on the same day I'd just caught the AI fabricating things twice and felt pretty good about it. Search around today and you'll still find no shortage of pages introducing this line as "one of Nomura's famous sayings." If what's meant is "a saying Nomura valued and lived by," that's accurate. But what tends to survive in a reader's memory is "a saying Nomura created" — and with no ill intent or carelessness required from anyone, the attributed origin of a good line drifts toward whoever is the most famous voice associated with it. Human memory isn't a recording being replayed; it's reconstructed every single time. And the first thing to drop out of that reconstruction is "where did I hear this" — a well-documented weak point in the area psychology calls source monitoring https://pubmed.ncbi.nlm.nih.gov/8346328/ . The content survives, but the source doesn't, so a good line drifts toward whichever speaker is the most famous. What AI does with plausible-sounding completions is, mechanically, remarkably close to what humans already do. What differs is scale, speed, and fluency. Which means the countermeasures can be the same ones used for fact-checking a human. No special AI-specific defense is required — just the same unglamorous procedures that have always existed, applied without skipping steps. Here are the four rules I've put into practice: Catching both of these in the same day wasn't because I'm smarter than the AI — it's simply that this one narrow domain talk around investing happens to be somewhere I have some accumulated ground-level familiarity from long exposure. Which also means: in a field where I don't have that familiarity, running this checklist mechanically is the only defense there is. If I had to compress the reviewer's posture into one line, it would be this: Whenever a passage is landing a little too smoothly, that's exactly when to ask, "wait, where did that actually come from?" Because the spot where prose lands most cleanly is the same spot where fabrication slides in most easily. One more disclosure, since it would be embarrassing if this very article fell into the trap it's describing: every quoted AI output referenced above was cross-checked against the actual logs from that day, and the Matsuura Seizan attribution was cross-checked against multiple sources, before publication. And yes — this article, too, was written in collaboration with AI and reviewed by a human. The review process caught things to fix here as well, not zero of them. Originally published in Japanese on Zenn: https://zenn.dev/idonthaveapen/articles/ai-fabricates-maxims-caught-twice