Humanizer has about 60,000 stars. On 12 public tasks my linter plugin scored 2.67 higher

A developer benchmarked a custom linter packaged as a Claude Code plugin against six other AI-writing-removal skills and a no-plugin baseline across 11 arms, running each task three times on Sonnet 5.5. On 12 public tasks the linter scored 58.67 of 60 versus 56.00 for the best rival (Humanizer invoked by name), a 2.67-point gap with a 95% bootstrap interval of [1.00, 4.33], but on 6 held-out tasks it trailed unslop by 0.33 and four arms clustered at 29.67 or above, so that set could not separate the top tools.

I wanted to know whether a linter for AI writing tells helps an agent rewrite a file better than a good prompt does. So I ran a benchmark with 11 arms: the linter packaged as a Claude Code plugin, six other Claude Code skills that claim to remove AI-sounding prose, and Claude with no plugin. Three of those skills Humanizer, unslop and everwrite were also run a second time with the prompt calling them by name, which is where the count of 11 comes from. Each arm ran each task three times on Sonnet 5.5. The short result: on 12 public tasks the linter scored 58.67 of 60 and the best other arm scored 56.00. On 6 held-out tasks, written by an author who had not read the code, four arms scored 29.67 or more of 30, so that set does not separate them. Each task asks an agent to edit a real file in place: a French launch note, a README intro, a pull request description, release notes, a mixed-language guide, a repository set-up. A task scores only what the file contains afterwards, so talking about the problem earns nothing. Each task also carries facts a careless rewrite would break, such as numbers, names, code and links, and scripted checks look for them. Some criteria need judgment, and those went to a blind grader: outputs of all arms were shuffled and anonymised, and the grader gave its reason before its verdict, five votes per cell. Of 99 blind cells, 92 were unanimous. The other six tools are Humanizer blader/humanizer, about 60,000 GitHub stars per the GitHub API on 2026-10-24 , unslop, everwrite, slopless, avoid-ai-writing and slop-cop. The protocol was written and committed before the first cell ran. A total below is the sum over tasks of the mean over the three runs, with a maximum of 5 points per task. The result applies to those tasks only and ranks nothing in general. | arm | total of 60 | 95% interval | |---|---|---| | this linter | 58.67 | 57.67, 59.67 | | Humanizer, called by name | 56.00 | 54.67, 57.33 | | Humanizer | 54.67 | 53.33, 56.33 | | everwrite, called by name | 54.33 | 52.67, 56.33 | | everwrite | 53.33 | 52.00, 54.67 | | unslop, called by name | 51.67 | 51.00, 53.00 | | unslop | 51.00 | 50.33, 51.67 | | slopless | 50.67 | 49.33, 52.00 | | avoid-ai-writing | 50.33 | 49.33, 51.67 | | slop-cop | 49.00 | 47.67, 50.33 | | no plugin | 48.67 | 47.00, 50.00 | The difference between the linter and the best other arm, Humanizer called by name, is 2.67 points, with a 95% bootstrap interval of 1.00, 4.33 . The lower bound is above zero. It did not win every task. On h11 and h15 it scored 4.67 where the best rival scored 5.00, and on h18 it tied at 4.67. The benchmark's stricter per-task rule rule B fails on those three tasks. Six further tasks were written by an author, a Sonnet agent, who had seen no code of this repository. The person running the measurement had not read them before it. | arm | total of 30 | 95% interval | |---|---|---| | unslop | 30.00 | 30.00, 30.00 | | avoid-ai-writing | 29.67 | 29.00, 30.00 | | this linter | 29.67 | 29.00, 30.00 | | unslop, called by name | 29.67 | 29.00, 30.00 | | no plugin | 28.67 | 28.00, 29.00 | | Humanizer | 28.67 | 28.00, 29.00 | | Humanizer, called by name | 28.00 | 27.00, 29.00 | | slopless | 28.00 | 28.00, 28.00 | | everwrite, called by name | 27.33 | 26.33, 28.33 | | slop-cop | 27.00 | 25.67, 28.33 | | everwrite | 26.67 | 26.00, 27.00 | Here the linter is 0.33 behind unslop, with an interval of -1.00, 0.00 , so the claim that it is first does not hold on this set. It lost that 0.33 on one task, ho1, where it scored 4.67 against 5.00. Four arms sit at 29.67 or higher and even the no-plugin baseline reaches 28.67, so these tasks cannot separate the top arms. Across both sets: first on the public set, and within 0.33 of the best of the 11 arms on the held-out set. I also ran Haiku 5.5 and Opus 5.5 once per task, with only the linter, the best other arm of each set and the no-plugin baseline. These totals are descriptive. With one run per task, a difference of one or two points is inside the run-to-run variation. These runs were not blind-graded, so the public scores use the scripted criteria only and their maximum is 57. | model | set | this linter | best other arm | no plugin | |---|---|---|---|---| | Haiku 5.5 | public, of 57 | 54 | 52 Humanizer, called by name | 43 | | Haiku 5.5 | held-out, of 30 | 28 | 28 unslop | 26 | | Opus 5.5 | public, of 57 | 54 | 54 Humanizer, called by name | 45 | | Opus 5.5 | held-out, of 30 | 29 | 27 unslop | 25 | The linter is a command line tool that calls no model and has no dependencies. Its rules are deterministic: the same text gives the same findings. A rewrite can pass every check and still lose a fact, so the tool also has --compare . It lists each number, version, date, link, quotation, code span and name that appears in one text and not the other, and exits 1 when it finds one. It also reports a qualifier that went missing next to a number, and first-person or claim words that the rewrite added. This is real output on a two-line example I wrote for this post: bash $ npx ai-slop-linter --compare before.md after.md facts: 1 kept, 4 lost, 0 added lost date 3 March 2026 lost number 1200 lost qualifier up to 40 lost version 2.4 The plugin's skill runs this after each rewrite and treats a failing comparison as blocking. --compare compares facts and not wording, so it cannot say whether a rewrite is good. This arm cost 2.43 times the no-plugin baseline per cell at list price, on both sets, because it takes more turns and more tokens. In the public set that was 0.125 US dollars per cell against 0.051. The runs were charged to a subscription, and the whole measurement would cost about 91 US dollars at list price. usage limit reached , were kept and never scored, and the same commands ran them again. npx ai-slop-linter README.md npx ai-slop-linter --compare before.md after.md To install it as a Claude Code plugin: /plugin marketplace add Bubblegunn/ai-slop-linter /plugin install ai-slop-linter@ai-slop-linter Every cell, the departures from the protocol and a recompute step are in the round 8 evidence folder https://github.com/Bubblegunn/ai-slop-linter/tree/main/bench/head-to-head/results/round8-final-2026-10-23 . REPRODUCE.md https://github.com/Bubblegunn/ai-slop-linter/blob/main/bench/head-to-head/results/round8-final-2026-10-23/REPRODUCE.md in that folder gives the commands for a re-run, which needs Claude Code 2.1.308, Node 24 and about 9 hours. The tasks, criteria and harness are public in bench/head-to-head/ , and the earlier rounds and rulings are in docs/HEAD-TO-HEAD.md https://github.com/Bubblegunn/ai-slop-linter/blob/main/docs/HEAD-TO-HEAD.md . The repository is Bubblegunn/ai-slop-linter https://github.com/Bubblegunn/ai-slop-linter , and version 0.2.0 is on npm.