{"slug": "our-great-mutator", "title": "Our Great Mutator", "summary": "A new analysis argues that large language models (LLMs) function as a 'mutator' or fuzzer, producing rubbish by design and requiring an oracle program to verify correctness, with the key benchmark being the number of tokens expended to converge to a mathematically correct output. The author, who proposes renaming generative AI to 'Our Great Mutator,' states that LLMs are useful only when paired with a correctness oracle, such as in fuzzing to crash programs or step-wise proof checking in Lean or Rocq, and have limited utility in non-critical annotation or synthesis tasks like translation or meeting summaries, while being actively harmful for critical functions like medical data annotation or customer support.", "body_md": "We argue that the sum total of everything LLMs do is a form of mutation or fuzzing, and that exactly one hard benchmark will suffice: number of tokens expended to converge to the mathematically correct output.\n\nWe certainly don't have heated arguments about `tar`\n\n, `ls`\n\n, or any other computer program that takes some input and produces some output. The difference is that the program in question produces something \"plausible\" immediately. The fact that the program does not have even a rudimentary logic engine by design necessarily means that it cannot mathematically print `2`\n\nwhen given `1 + 1`\n\nas the input. In fact, the only possible way for this program to print the correct output immediately is to quote a known-correct output verbatim, by chance. In other words, an LLM always produces *rubbish*, by design. The output space of this program is necessarily a *mutation* of its training data and the supplied prompt; it cannot *compute* or *construct* anything.\n\nThat said, the program is remarkably good at iterative mutation; on each iteration, it successively produces less rubbish, and converges to a mathematically correct output faster than any other software. The program is intended to be used along with an oracle program that mechanically checks for correctness. In the absence of the availability of an oracle program, a human has to act as the oracle, and this is the most painful and least effective way to use the program. The program has limited utility in non-critical tasks, mutating to *synthesize* output well within its training data, and to *annotate* outliers that fall in the periphery of its training data. Otherwise, the program is useless.\n\nSome good uses of the mutator (correctness oracle available):\n\n- Fuzzing to find an input to crash a user program. A crash is a crash: this is the correctness criterion, and the oracle is the user program.\n- Fuzzing to find an input to make a user program print an unsound output. In this case, we need a known-correct variant of the program whose output the LLM can compare against, or some other kind of mathematical oracle program like an SMT solver.\n- Fuzzing to find a proof, step-wise, when working in a proof assistant like Lean or Rocq. Proof assistants are powerful mathematical oracles that allow users to do a kind of mechanized mathematics, step-wise. They check each step in the proof, and simply report \"correct\" or \"incorrect\". This is an especially good use case, and the mutator can be used to \"solve open mathematical problems\".\n\nSome limited uses of the mutator (non-critical annotation/synthesis):\n\n- Annotating outlier behavior from surveillance camera input. No guarantee catching a serious outlier, useful to the human watching nevertheless.\n- Annotating outlier grammar in natural language input. Non-critical, better than other programs.\n- Synthesizing a translation of a natural language input. Non-critical, better than other programs.\n- Synthesizing a humanoid voice to do text-to-speech. Non-critical, better than other programs.\n- Synthesizing a summary of corporate meetings. Non-critical, better than nothing.\n\nSome non-uses of the mutator:\n\n- Generating a piece of creative writing or music. The piece will have no audience.\n- Generating code that passes tests. The code in question is sure to be convoluted and buggy.\n- Annotating outlier medical data. Critical function, actively harmful.\n- Generating responses to customer support queries. The company in question is sure to lose business.\n- Generating prose to replace an internet search engine. Cannot reproduce excerpts verbatim, by design.\n\n\"Generative AI\" is really poor at generation tasks. I propose renaming it to \"Our Great Mutator\".", "url": "https://wpnews.pro/news/our-great-mutator", "canonical_source": "https://artagnon.com/art/mutator", "published_at": "2026-07-30 09:00:25+00:00", "updated_at": "2026-07-30 09:22:42.828090+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-safety", "ai-tools"], "entities": ["Lean", "Rocq"], "alternates": {"html": "https://wpnews.pro/news/our-great-mutator", "markdown": "https://wpnews.pro/news/our-great-mutator.md", "text": "https://wpnews.pro/news/our-great-mutator.txt", "jsonld": "https://wpnews.pro/news/our-great-mutator.jsonld"}}