{"slug": "lint-your-technical-drafts-for-ai-writing-tells-before-you-publish", "title": "Lint your technical drafts for AI writing tells before you publish", "summary": "A developer has published a two-pass method for catching AI-writing tells in technical drafts, pairing a small Python regex sketch that flags vague attribution, contrast frames, generic conclusions and significance inflation with a manual pass that requires each flagged claim to be backed by something verifiable. The approach draws on Chain of Thought episode 54, in which Scaled Cognition co-founder and CTO Dan Klein argues LLMs are \"plausibility engines\" that produce likely rather than verified text, and the writeup stresses that linting fixes prose without proving the underlying engineering claim.", "body_md": "A draft about your retrieval layer can read like a launch announcement and still say nothing you could test. Checking the prose is cheap. Checking the engineering claim is a different job. I treat AI-writing lint the way I treat a style checker on code: fast feedback on patterns, and zero proof that the logic is right.\n\nA [large language model](https://chainofthought.show/glossary/large-language-model/) is trained to produce likely text, not verified text, so a polished explanation can be wrong and still read like documentation. Dan Klein, co-founder and CTO of Scaled Cognition, described the problem in his episode: older language technology often showed its mistakes, because a translation that read badly also tended to be unfaithful. A technology that always sounds right but often is not makes it hard to know what to trust.\n\nLike I said, one of my biggest fears is that, as a field, we are building systems which produce outputs indistinguishable from the truth.\n\n*Dan Klein, on Chain of Thought ep 54*\n\nThe Chain of Thought writing page makes the same point with a made-up paragraph about a retrieval layer. It marks four patterns from the open-source avoid-ai-writing catalog, and then notes that fixing them makes the paragraph read better while telling you nothing about whether the retrieval layer reduces hallucinations.\n\nThe page labels these:\n\nWhen you edit, highlight every instance. If you cannot attach something verifiable on the next pass, cut the sentence or rewrite it into a claim a reviewer could prove wrong.\n\nTwo passes on a technical post, in this order:\n\nOn a long post I would rather let a script find the pattern families first. This sketch is mine, and deliberately small; a maintained detector is the better tool for real use.\n\n``` python\n# Sketch: flag a few AI-writing tells in a markdown draft\nimport re\nfrom pathlib import Path\n\ntext = Path(\"draft.md\").read_text(encoding=\"utf-8\")\nrules = [\n    (r\"\\b(experts agree|studies show|researchers say)\\b\", \"vague attribution\"),\n    (r\"\\bnot (just|only|merely)\\b[^.]*\\bit'?s\\b\", \"contrast frame\"),\n    (r\"\\bthe future (looks|is)\\b\", \"generic conclusion\"),\n    (r\"\\b(watershed|sea change|in the evolution of)\\b\", \"significance inflation\"),\n]\nfor pattern, label in rules:\n    for m in re.finditer(pattern, text, re.I):\n        line = text.count(\"\\n\", 0, m.start()) + 1\n        print(f\"{label} ~ line {line}: {m.group()[:60]}\")\n```\n\nTreat the hits like linter warnings: review each one, because some will be false positives, and fix the real ones.\n\nA detector only sees shape. Removing an unnamed expert does not add evidence. Replacing a claim of historic weight with a concrete before and after does, but only if the before and after are real.\n\nThat is why the writing page sits next to three episodes on the gap between plausible and true: Dan Klein on plausibility engines (episode 54), Sudhir Hasbe on hallucinations as a data architecture problem (episode 55), and Alex Ratner on the evaluation gap, where everyone says their system works and almost nobody can say what \"works\" means in a number (episode 57). Lint gets you to honest prose. Those conversations are about what honest prose is still allowed to claim.\n\nIf you want to see the patterns in a real draft, the [essays](https://chainofthought.show/essays/) include a piece where an AI agent's first draft runs with formatting fixes only, so you can read it for tells yourself. More about the show and how it is made is on the [about page](https://chainofthought.show/about/).\n\nMore on this topic, with the related episodes, is on [Chain of Thought](https://chainofthought.show/ai-writing/). It draws on [this episode](https://chainofthought.show/podcast/54-why-llms-are-plausibility-engines-not-truth-engines-dan-klein/).\n\n[Subscribe to the Chain of Thought newsletter](https://newsletter.chainofthought.show/subscribe?utm_source=devto&utm_medium=referral&utm_campaign=newsletter) for new episodes and write-ups like this one.\n\n*Drafted with AI assistance from the episode transcripts.*", "url": "https://wpnews.pro/news/lint-your-technical-drafts-for-ai-writing-tells-before-you-publish", "canonical_source": "https://dev.to/conorbronsdon/lint-your-technical-drafts-for-ai-writing-tells-before-you-publish-1jg2", "published_at": "2026-10-28 20:30:07+00:00", "updated_at": "2026-10-28 20:41:21.781652+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "developer-tools"], "entities": ["Chain of Thought", "Dan Klein", "Scaled Cognition", "Sudhir Hasbe", "Alex Ratner"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/lint-your-technical-drafts-for-ai-writing-tells-before-you-publish", "markdown": "https://wpnews.pro/news/lint-your-technical-drafts-for-ai-writing-tells-before-you-publish.md", "text": "https://wpnews.pro/news/lint-your-technical-drafts-for-ai-writing-tells-before-you-publish.txt", "jsonld": "https://wpnews.pro/news/lint-your-technical-drafts-for-ai-writing-tells-before-you-publish.jsonld"}}