{"slug": "a-little-experiment-in-evading-ai-detection", "title": "A little experiment in evading AI detection", "summary": "A developer used Claude Code and 10 parallel AI agents to rewrite 84 blog posts in an attempt to evade Pangram's AI detection, but all posts were still flagged as AI-generated, with scores ranging from 0.77 to 1.00. The experiment demonstrates the difficulty of bypassing current AI detection tools even with iterative feedback from the detector itself.", "body_md": "# A little experiment in evading AI detection\n\n**TLDR:** I tried to evade AI detection on a heavily AI written corpus using Claude Code, 10 parallel agents, and some basic tricks. It didn't work, but it was fun.\n\n*Disclaimer: I am in no way affiliated with the people behind Pangram, but kudos to them for making something that seems to do what it says. You will have to be more clever than me to evade them.*\n\nAs a learning exercise I used AI to write the series [Holding the LLM Stack in Your Head](/series). It was developed in collaboration with Claude Code and Opus 4.6/4.7/4.8. Creating it was fantastically fun, and I learned a ton. I'm going to do something like that again on a different topic, and you should too! You will learn something.\n\nAnyway, I was pretty sure that AI detection would flag a good chunk of it, but not all of it. I had thousands of lines of back and forth with Claude, massaging the posts until they were in a state that pleased me. My assumption was that all the *genuine human tokens* I input into Claude Code would influence the output tokens enough to come across as human, or at least a *little* bit human.\n\nI was wrong. After getting an API key to Pangram and evaluating all the prose in the 84 posts, almost every single one was flagged as 100% AI. This gave me the idea to test if I could evade that AI detection using the *same AI* that was used to generate that text.\n\nFirst some quick stats. Thanks for the visual Claude.\n\n## Round 1\n\nMy first pass at evading the AI detector wasn't very clever. I pulled down the full text of this Wikipedia page, [Signs of AI Writing](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing). I passed all those tokens into the context for each of 10 subagents, one for each arc, that were instructed to use the information in the wiki to edit each post in their arc in a way that made it sound like human writing and not AI.\n\nHere are a few examples of things it did. Again, thanks Claude for making the visual.\n\nThe edits were pretty minor, but theoretically guided by the information in the wiki. They do seem to be a bit \"less AI\" to me.\n\nBut yeah, as you probably guessed based on all my framing: this didn't work at all. Every post was basically still flagged as 100% AI generated.\n\n## Round 2\n\nRound two I tried to get more clever. I gave each agent access to a script that allowed them to safely call the Pangram API on the prose of the blogs they were assigned. They were also given the wikipedia article and instructed to iterate on the content of their arc, retrying the API and seeing their AI score, until they were able to get a human label or they ran out of the credits the script assigned them.\n\n| Arc | Topic | Posts | Words submitted | `fraction_ai` |\n|---|---|---|---|---|\n| 1 | Math prerequisites | 8 | ~27,000 | 0.92 – 1.00 |\n| 2 | Pre-transformer LMs | 7 | ~20,760 | 0.87 – 1.00 |\n| 3 | Tokenization | 7 | ~23,700 | 0.87 – 1.00 |\n| 4 | Transformers | 9 | ~28,478 | 0.85 – 1.00 |\n| 5 | Decoding | 9 | ~30,500 | 0.87 – 1.00 |\n| 6 | Inference engines | 9 | ~26,650 | 0.89 – 1.00 |\n| 7 | Training | 10 | ~33,970 | 0.88 – 1.00 |\n| 8 | Evaluation | 7 | ~24,400 | 0.77 – 1.00 |\n| 9 | Retrieval | 9 | ~29,100 | 0.93 – 1.00 |\n| 10 | Agents | 9 | ~29,500 | 0.92 – 1.00 |\nTotal | 84 | ~274,000 | all \"AI Detected\" |\n\nDistribution of the 84: **~62** posts scored 0.99–1.00, **~15** scored 0.90–0.98, and **~7** scored 0.77–0.89. The single least-AI post was `contamination`\n\nat 0.77. None of them earned the label \"human\".\n\nIt was interesting that the AI couldn't evade the AI detection even when given multiple accesses to the AI detecting oracle. I suppose it is possible that I just didn't let it iterate enough, but I didn't wanna waste money. If I find a good open weight AI detection model I'll give it a try in a loop while I sleep.\n\nHowever, if you inspect the edits, they aren't very ambitious edits. I'll address that in a sec.\n\nYou see, Claude Code was using its file editing tool to do the rewriting, which I think biased it toward keeping in place the existing structure. Sentence length, flow, structure, and other longer range patterns are probably things that the detection is keying in on.\n\nSo let's have it not use the file edit tool and instead create a whole new file from scratch each time.\n\n## Well, turns out that STILL didn't work.\n\n## A control\n\nAt this point I wanted to do a quick control test to make sure the detector works on human authored blogs. I admit all this is probably labeled as human inside their very own training data, so this isn't really telling but I did it anyway to be sure.\n\nPulled the blogs from these famous tech authors and they all got 0%.\n\n| Human post (900-word slice) | Author, year | `fraction_ai` |\n|---|---|---|\n|\n\n**0.00**[The Unreasonable Effectiveness of RNNs](https://karpathy.github.io/2015/05/21/rnn-effectiveness/)** 0.00**[The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/)** 0.00**[Attention? Attention!](https://lilianweng.github.io/posts/2018-06-24-attention/)** 0.00**[An overview of gradient descent](https://www.ruder.io/optimizing-gradient-descent/)** 0.00**Claude, a visual please.\n\nNice.\n\n## Making it mimic a voice\n\nI did some digging and apparently students are having some success by asking the model to mimic the voice of a specific person. I'm guessing because that person in literally in the training data of the AI detector and labeled human, so if we can mimic some of those features we can evade. Unless of course they did some adversarial robustness training, which honestly would be really easy to do in this specific domain. Let's see.\n\nI picked [Andrej Karpathy](https://karpathy.github.io/), downloaded 8 of his tech blogs and gave each subagent the context. They were to mimic his style and voice.\n\nVisual:\n\n## One thing worked\n\nI made [a post](/blog/my-throw-decides-my-aim) earlier and had AI help me edit it. It was a genuinely me thought, but I got caught up wanting to make myself sound smarter, so I let AI take a pass at editing the chicken scratch of an idea I had for a little post. In the end I got a post that lost my voice entirely and was way too long.\n\n84% AI Generated according to the API btw.\n\nI thought, why put this out there at all if I didn't write it?\n\nSo I just took some time to write it myself. 0% AI. Human label.\n\nNice. Claude, one more visual.\n\nThis post btw:\n\nHuman Written. fraction_ai 0.00, fraction_human 1.00", "url": "https://wpnews.pro/news/a-little-experiment-in-evading-ai-detection", "canonical_source": "https://thegustafson.com/blog/evading-ai-detection", "published_at": "2026-07-18 05:40:16.378251+00:00", "updated_at": "2026-07-18 05:40:18.416078+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "ai-ethics"], "entities": ["Claude Code", "Pangram", "Opus"], "alternates": {"html": "https://wpnews.pro/news/a-little-experiment-in-evading-ai-detection", "markdown": "https://wpnews.pro/news/a-little-experiment-in-evading-ai-detection.md", "text": "https://wpnews.pro/news/a-little-experiment-in-evading-ai-detection.txt", "jsonld": "https://wpnews.pro/news/a-little-experiment-in-evading-ai-detection.jsonld"}}