Problematic Paper Screener detects 1000s of fraudulent/slop scientific articles

A tool built by computer scientists at Université de Toulouse and Université Grenoble Alpes has flagged nearly 19,000 scientific papers containing at least five 'tortured phrases'—nonsensical word sequences generated by paraphrasing tools to evade plagiarism detection—along with more than 280 gibberish papers written by the MIT-created SCIgen program, over 764,000 articles citing retracted works, and more than 70 papers with obvious ChatGPT fingerprints. The Problematic Paper Screener, developed by Guillaume Cabanac, combs through 130 million publications weekly and has contributed to over 1,000 retractions, with several publishers integrating it into their editorial workflows.

Have you ever heard of the Joined Together States https://pubpeer.com/search?q=%22Joined+Together+States%22 ? Or bosom peril https://pubpeer.com/search?q=%22bosom+peril%22 ? Kidney disappointment https://pubpeer.com/search?q=%22Kidney+disappointment%22 ? Fake neural organizations https://pubpeer.com/search?q=%22Fake+neural+organization%22 ? Lactose bigotry https://pubpeer.com/search?q=%22Lactose+bigotry%22 ? These nonsensical, and sometimes amusing, word sequences are among thousands https://www.irit.fr/%7EGuillaume.Cabanac/problematic-paper-screener/fingerprints of “ tortured phrases https://doi.org/10.48550/arXiv.2107.06751 ” that sleuths have found littered throughout reputable scientific journals. They typically result from using paraphrasing tools to evade plagiarism-detection software when stealing someone else’s text. The phrases above are real examples of bungled synonyms for the United States, breast cancer, kidney failure, artificial neural networks, and lactose intolerance, respectively. We are a pair of computer scientists at Université de Toulouse https://www.irit.fr/%7EGuillaume.Cabanac/ and Université Grenoble Alpes https://membres-lig.imag.fr/labbe/ , both in France, who specialize in detecting bogus publications. One of us, Guillaume Cabanac, has built an automated tool that combs through 130 million scientific publications every week and flags those containing tortured phrases. The Problematic Paper Screener https://www.irit.fr/%7EGuillaume.Cabanac/problematic-paper-screener also includes eight other detectors https://doi.org/10.1002/asi.24495 , each of which looks for a specific type of problematic content. Several publishers use our paper screener, which has been instrumental in more than 1,000 retractions. Some have integrated the technology into the editorial workflow to spot suspect papers upfront. Analytics companies have used the screener for things like picking out suspect authors from lists of highly cited researchers. It was named one of 10 key developments in science https://doi.org/10.1038/d41586-021-03621-0 by the journal Nature in 2021. So far, we have found: Nearly 19,000 papers containing at least five tortured phrases each. More than 280 gibberish papers – some still in circulation – written entirely by the spoof SCIgen program https://pdos.csail.mit.edu/archive/scigen/ that Massachusetts Institute of Technology students came up with nearly 20 years ago.More than 764,000 articles that cite retracted works that could be unreliable. About 5,000 of these articles have at least five retracted references listed in their bibliographies. We called the software that finds these the “Feet of Clay” detector https://doi.org/10.1038/d41586-024-02747-1 after the biblical dream story where a hidden flaw is found in what seems to be a strong and magnificent statue. These articles need to be reassessed https://doi.org/10.1038/d41586-024-02719-5 and potentially retracted.More than 70 papers containing ChatGPT “fingerprints” with obvious signs such as “ Regenerate Response https://pubpeer.com/search?q=%22Regenerate+Response%22+AND+ChatGPT ” or “As an AI language model, I cannot …” https://pubpeer.com/search?q=%22As+an+AI+language+model%2C+I%22 in the text. These articles represent the tip of the tip of the iceberg: They are cases where ChatGPT output has been copy-pasted wholesale into papers without any editing or even reading and has also slipped past peer reviewers and journal editors alike. Some publishers allow the use of AI to write papers, provided the authors disclose it. The challenge is to identify cases where chatbots are used not just for language-editing purposes but to generate content – essentially fabricating data. There’s more detail about our paper screener and the problems it addresses in this presentation https://youtu.be/P2mr8N OGs?feature=shared for the Science Studies Colloquium https://scientificelites.org/scistud/ . Read The Conversation’s investigation into paper mills here: Fake papers are contaminating the world’s scientific literature, fueling a corrupt industry and slowing legitimate lifesaving medical research