{"slug": "detecting-ai-ideas-not-ai-text", "title": "Detecting AI Ideas, Not AI Text", "summary": "A University of Maryland-led academic collaboration developed IdeaLens, a detection method trained on one million outlines extracted from web documents that identifies whether the ideas in a document originated with AI regardless of who wrote the final text. The approach reduces each document to an outline that preserves its ideas while stripping away wording, and an online demo at ideadetector.ai distinguishes human ideas from human prose and AI ideas from AI prose using detectors trained on the same documents, labels and model architecture. The researchers attribute the traceable signature to LLMs drawing on statistically dominant concepts in under-curated training data.", "body_md": "### \n[Anderson's Angle](https://www.unite.ai/series/andersons-angle/)\n\n# Detecting AI Ideas, Not AI Text\n\n[Add Unite.AI to your preferred sources on Google](https://www.google.com/preferences/source?q=unite.ai)\n\nWhat if there was an AI text detector that could recognize that you wrote something based on an *idea* that AI gave you – even if all the actual writing (i.e., the text) was your own original work?\n\nWe all know about AI text detectors embarrassing [celebrities](https://www.indiatoday.in/technology/news/story/cristiano-ronaldo-admits-he-was-nervous-before-the-prestige-globe-award-turned-to-perplexity-ai-for-help-2799716-2025-10-08) and [politicians](https://halifax.citynews.ca/2026/07/29/video-of-new-brunswick-politicians-apparent-ai-use-goes-viral/), among others – algorithms that have studied the characteristics of AI-created text and can [discern](https://www.unite.ai/predicting-violence-in-advance-with-ai/#:~:text=can%20pick%20out%20that%20characteristic%20on%20similar%20data%20that%20was%20not%20used%20in%20training) those patterns when they surface.\n\nThese cases represent simple offloading of a communications task, wherein language models such as [ChatGPT](https://www.unite.ai/how-to-use-openais-chatgpt-agent-a-step-by-step-guide/) and [Google Gemini](https://www.unite.ai/googles-multimodal-ai-gemini-a-technical-deep-dive/) [predict the next likely token](https://www.unite.ai/prompt-engineering-in-chatgpt/#:~:text=predicting%20the%20next%20sequence%20of%20likely%20tokens) in a response to a user’s prompt – not by taking a step back and looking at the broad canvas of the problem or proposition first, but just by guessing the next most probable word.\n\nNonetheless, this patchwork quilt of predictions does eventually yield an underlying structure, according to recent research: the need for adjacent text sections to be coherent and relevant to each other causes a unique signature to develop in the ‘outline’ or higher dimensionality of the work – [one that can discern the hand of AI](https://www.unite.ai/industrializing-humanization/) even when the text is extensively rewritten to ‘humanize’ it:\n\n## AI Ideation Targeted\n\nA new academic [collaboration](https://arxiv.org/pdf/2610.06778) led by the University of Maryland goes a step further, by offering a methodology that can detect whether the *ideas themselves* originated with AI, regardless of who ultimately wrote the words.\n\nThe approach, dubbed *IdeaLens* and trained on one million outlines extracted from web documents,  keys on reducing each document to an outline that preserves its *ideas* while stripping away the wording – and comes with an [online demo](https://ideadetector.ai/):\n\nAs seen in the sample demo results above, the detection methods used can distinguish between human ideas and human prose, and AI ideas and AI prose, and evaluate them in a granular and distinct fashion – even though the two detectors were trained on the same documents, labels and underlying model architecture, while being designed to detect essentially *opposite* things: the provenance of ideas, and the provenance of prose.\n\nThis kind of detection ambit, one suspects, could make a number of categories of writer nervous, from [political speech writers](https://www.economist.com/britain/2026/09/23/ai-written-speeches-are-taking-over-politics) through to [novelists](https://www.theguardian.com/books/2026/jul/31/crime-novel-deal-collapses-questions-ai-jerry-falade-call-me-ill-hide-the-body) and [columnists](https://www.washingtonpost.com/business/2026/08/25/wall-street-journal-says-ai-generated-op-ed-didnt-breach-its-standards/), besides others who have, to date, prized their own writing style, yet allowed AI to fuel their *topics*.\n\nOne reason why LLMs are likely to produce recognizable (and therefore traceable) ideas is not just that certain concepts and texts will have statistically dominated the model’s [under-curated training data](https://www.unite.ai/are-under-curated-hyperscale-ai-datasets-worse-than-the-internet-itself/), making them more likely to surface in a wider range of requests; but also, because LLMs have mysterious and repetitive obsessions that don’t even seem to relate to specific training data, and yet [often appear unbidden](https://www.unite.ai/why-does-ai-love-writing-about-lighthouse-keepers/).\n\nThe authors state*:\n\n*‘While modern AI detectors identify **who wrote the words**, emerging policies on AI use increasingly hinge on a different question: **who came up with the ideas?** We introduce IdeaLens, a detector that identifies whether a document’s ideas came from a human or AI (idea provenance), regardless of who wrote its words.* \n\n*‘To focus IdeaLens on ideas rather than prose, we represent documents as outlines: lists of items that each pair a discourse role with a brief, paraphrased description of the content, minimizing word-level overlap with the raw text.’*\n\n## Data and Methods\n\nThe paper claims that the new method can identify human ideas in AI-generated documents (i.e., someone said to an AI ‘*Write me an essay on this topic’*); and can also identify AI ideas in human writing (i.e., someone said to an AI *‘Suggest something for me to write about’*).\n\nDetection is easiest when provenance is clearest – for example, when a human-written piece is based on AI-generated ideas, or where an AI-generated output originates from a human idea. When these provenance strengths vary, detection becomes harder.\n\nThe researchers tested this distinction by gradually changing how much of a document’s underlying plan comes from a person. In one experiment, AI models were given increasingly detailed human plans, ranging from not much more than a topic, to a complete outline.\n\nAs the human contribution increased, IdeaLens’s AI flag rate fell from 94.9% to 6.8%; and conventional prose detectors continued overwhelmingly to identify the resulting text as AI:\n\nSince the ideas must be separated from the language used to express them, and there is no large dataset recording who actually originated the ideas in a document, the researchers instead trained IdeaLens on existing AI-writing [*labels*](https://www.unite.ai/computer-vision-image-annotation-services-data-labeling-quality/) – but removed most of the writing itself:\n\nAs shown above, each document was reduced to an outline describing what is being said, and the role each point plays in the document. During training, those outlines were paraphrased to further remove traces of the authors’ original wording. IdeaLens can therefore make its predictions largely from the *ideas that remain*, rather than from telltale characteristics of the prose.\n\nThree datasets were originated for the project: *IdeaShift*; *IdeaShift-X*; and *TwiceTold*. IdeaShift was created from 500 human-written and AI-generated seed documents sampled from the researchers’ existing test corpus. GPT-5.6 Sol to turn each into progressively more detailed prompts, from ‘topic-only’, to a full outline. [GPT-5.6](https://www.unite.ai/openai-brings-gpt-5-6-model-family-to-awss-kiro/) was then used to generate new documents.\n\nIdeaShift-X, instead, used 10,000 human-authored [FineWeb2](https://arxiv.org/abs/2506.20920) documents across 24 languages, applying the aforementioned IdeaShift protocol to generate AI versions in each language.\n\nThe third dataset, TwiceTold, was written by academic students under the supervision of the authors, to avoid the possible use of AI [inherent](https://www.unite.ai/adobe-and-meta-decry-misuse-of-user-studies-in-computer-vision-research/#:~:text=Handling%20Crowdworkers%20Who%20Cheat) in crowd-sourcing such data. Fifty stories were written from scratch, but using a 500-word outline provided by ChatGPT-5.6 Sol.\n\n## Tests\n\nThe researchers’ own detectors comprised IdeaLens in outline and document form; the aforementioned ProseLens; [ModernBERT](https://github.com/AnswerDotAI/ModernBERT)-L versions of both systems; [Qwen3.5-9B](https://www.xda-developers.com/qwen-3-5-9b-tops-ai-benchmarks-not-how-pick-model/) versions of IdeaLens; and an IdeaLens [logistic-classifier](https://www.ibm.com/think/topics/logistic-regression) variant.\n\nThese were compared against [Pangram 4](https://www.pangram.com/research/model-card/pangram-4); [EditLens-Llama-3B](https://huggingface.co/pangram/editlens_Llama-3.2-3B); [Binoculars](https://openreview.net/pdf?id=axl3FAkpik); [EditLens-RoBERTa](https://huggingface.co/pangram/editlens_roberta-large); [MELD](https://arxiv.org/abs/2605.06903); [Desklib-academic](https://huggingface.co/desklib/ai-text-detector-v1.01); [Desklib](https://huggingface.co/desklib/ai-text-detector-v1.01); [Entropy](https://aclanthology.org/P19-3019/); [Fast-DetectGPT](https://openreview.net/pdf?id=Bpcgcr8E8Z); [Likelihood](https://aclanthology.org/P19-3019/); [Log-rank](https://proceedings.mlr.press/v202/mitchell23a.html); [LRR](https://aclanthology.org/2023.findings-emnlp.827/); [MAGE](https://aclanthology.org/2024.acl-long.3/); [OpenAI-RoBERTa-base](https://arxiv.org/abs/1908.09203); [OpenAI-RoBERTa-large](https://arxiv.org/abs/1908.09203); [RADAR](https://openreview.net/forum?id=Bpcgcr8E8Z); and [Rank](https://arxiv.org/abs/1811.12231).\n\nFor the tests, Accuracy was measured by whether the predicted AI or human label matched the source of the ideas. IdeaLens, ProseLens and EditLens used a global 1% FPR threshold; Pangram 4’s AI, AI-assisted and human labels were binarized; and Fast-DetectGPT, Binoculars, MELD and Desklib used their default thresholds.\n\nThe wider evaluation used 19 external benchmarks, including 14 covering fully human or fully AI material, and 10 containing human writing subsequently edited by AI. A further 50,000 in-domain documents were tested, while 10,000 pre-ChatGPT C4 documents were used to assess false positives:\n\nAcross 51 tests drawn from 22 benchmarks, IdeaLens was the only detector to perform strongly when the ideas and prose came from either the same or different sources, scoring 95.3% and 81.3% respectively.\n\nBy comparison, ProseLens and Pangram 4 scored above 98% when ideas and prose shared a source, but fell to around 25% when they did not.\n\nThe authors state of these results*:\n\n*‘Since ProseLens has the same backbone, training documents, and training labels as IdeaLens, this highlights the impact of our outline representation, which a [ModernBERT-based pair](https://arxiv.org/abs/2412.13663) replicates with a different backbone.* \n\n*‘IdeaLens also transfers unexpectedly well to raw text inputs at test time, exhibiting high shared provenance accuracy and substantially higher mixed provenance accuracy than ProseLens.’*\n\nAll three detectors successfully identified almost all of the original AI-written stories. On TwiceTold, IdeaLens identified AI-originated ideas in 68% of the human-written stories, compared to 0% for ProseLens and 8% for Pangram 4:\n\nDespite being trained in English, IdeaLens generalized across IdeaShift-X’s 24 languages, flagging 95.3% of documents based on AI ideas – but just 0.7% when a detailed human plan supplied the ideas. On 10,000 pre-ChatGPT human documents across the same languages, the false-positive rate was a mere 0.1%.\n\nThe authors concede that a larger dataset would be desirable in future work, and that the noisiness of the dataset labels could need addressing. Nonetheless, those that wish to build on this interesting initial outing can resort to the [GitHub repository](https://github.com/RishanthRajendhran/IdeaLens) that the paper’s authors have made available; and the merely-curious can paste ‘suspect’ tests into the project’s [demo site](https://github.com/RishanthRajendhran/IdeaLens), and gauge for themselves if the method jibes with their own estimations, or those of other approaches.\n\n## Conclusion\n\nWhether or not a detection method of this kind *matters* depends on how acceptance, integration, or rejection of AI will develop (no doubt differently across different sectors and circumstances) over the next 6-12 months.\n\nIt may be that a politician’s public are more forgiving of his or her use of AI to polish and represent their own original ideas, than they would be of a glib presentation of ideas generated by an LLM (after all, statistically, the audience is [likely to be sympathetic](https://www.pewresearch.org/data-labs/2026/08/20/how-much-of-the-internet-is-written-with-ai/) to the use of AI as a mere presentation aid).\n\nYet, at the same time, what are we hoping for from AI if not that it will produce angles, ideas and alternatives that we ourselves would have overlooked or never considered; and from that point of view, is it such a bad idea to give machine intelligence a turn at the wheel?\n\nWell, in terms of optics, and given AI’s famous and growing propensity for errors and –lately – [errant behavior](https://www.unite.ai/openai-says-its-agents-posted-53-user-images-to-image-hosting-sites/), maybe it’s still too early to let LLMs originate policy suggestions, or in general become a driving force in ideation.\n\n* *Authors’ emphases, not mine, but my conversion of the authors’ inline citations to hyperlinks, where necessary.*\n\n*First published Tuesday, October 6, 2026*", "url": "https://wpnews.pro/news/detecting-ai-ideas-not-ai-text", "canonical_source": "https://www.unite.ai/detecting-ai-ideas-not-ai-text/", "published_at": "2026-10-06 00:00:00+00:00", "updated_at": "2026-10-06 17:16:42.854379+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-safety"], "entities": ["University of Maryland", "IdeaLens", "ideadetector.ai", "ChatGPT", "Google Gemini"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/detecting-ai-ideas-not-ai-text", "markdown": "https://wpnews.pro/news/detecting-ai-ideas-not-ai-text.md", "text": "https://wpnews.pro/news/detecting-ai-ideas-not-ai-text.txt", "jsonld": "https://wpnews.pro/news/detecting-ai-ideas-not-ai-text.jsonld"}}