AI-generated writing: it’s still bad

AI-generated writing remains poor three years after widespread adoption of large language models, according to a blogger's analysis that found ChatGPT's imitation of Scott Alexander's style added 33 words without new substance and scored only 32% human on Pangram's detector. The author argues that LLMs' reliance on prediction and association works for math or coding but fails to capture the nuance and idiosyncrasies of human prose, making AI writing verbose and overly formal.

I saw this going viral, “ AI;DR AI; Didn’t Read https://www.rickmanelius.com/p/aidr-ai-didnt-read :” I’ve been thinking about it ever since. Why? Because there is growing grumbling among everyone about AI writing. And it’s not just others; it’s me I am getting to the point where I physically flinch sometimes dropping my shoulders and hunching, or having a slight eye twitch when someone I respect sends me unfiltered and unedited AI output. Yes, there are certain situations in which we should expect 100% AI-generated copy. Customer support would be a perfect example. We’re not looking for artisanal “did you make sure to reset your phone” style dialogue. It’s remarkable how in the 3 years since the widespread adoption of LLMs, AI-generated writing is still bad despite huge progress elsewhere, such as math, short video, or coding. AI writing ranges from overly wordy, to torturous, or too formal. It cannot reproduce the clear, effortless prose of professional, human-produced writing. Even for writing styles that are less concise or verbose, every word is necessary or serves a function, whereas AI inserts extra words. The dream of entering a prompt and a desired writing style, and the AI produces prose that is unrecognizable as Ai and passes as human, is apparently harder than advanced math. According the Pangram, the above two short paragraphs checks as “100% human:” I then asked ChatGPT rewrite the above in the style of blogger Scott Alexander, who is widely regarded as a competent writer and has a significant corpus of online material from which to draw from. Here is what it produced: It's remarkable that, three years into the widespread adoption of LLMs, AI-generated writing is still so bad, despite enormous progress in other areas: mathematics, coding, short-form video, and so on. AI prose tends to be either bloated, tortured, or oddly formal. It still struggles to reproduce the clear, effortless prose of a good human writer. Even writers whose style is deliberately verbose usually make every word do some work. AI has a tendency to add words simply because they sound like the sort of words that ought to be there. The result is prose that is longer without being richer. The dream was supposed to be simple: enter a prompt, specify a writing style, and get prose that is indistinguishable from something written by a human. Somehow, that problem has turned out to be harder than advanced mathematics. Confirming the verbosity and bloat, it added 33 words without actually producing anything new or substantive. It failed at imitating Scott’s style. It just sounds vacuous, without the inside-jokes or clever wordplay typical of his posts. According to Pangram, it checks as only 32% human: People could be forgiven for using AI if it wasn’t so bad or blatantly obvious, but I don’t see this changing, owing to the limitations of the underlying technology itself. LLMs use prediction and association. It’s doing what it’s supposed to: making inferences based on context. This works great for math or coding because aesthetics or style doesn’t matter: it need only to achieve the sought objective of whoever is entering the prompt. The end-user only sees the output, not all the machinery behind it. Only that it works is good enough. Writing is different because the human thought process is circuitous, messy, or erratic, and AI is trying to instead get from A to B, and in the process, glosses over the nuance or idiosyncrasies of what makes human writing “tick”. AI writing sounds too formal because it’s doing its job too well, or trying too hard. It adds extra words because it’s trying to be cautious based on some internal epistemological confidence meter, and in the process, the writing becomes bloated. It cannot just “say it”. BTW, the above text again checks as “100% human”.