{"slug": "science-or-slop", "title": "Science or Slop?", "summary": "The Science Slop Index, a tool from the project Science or Slop?, scores papers on six measures of scientific reasoning rather than token-level AI detection, reporting a result out of 100. The project reports pair accuracy on SciSlopBench, where 390 AI-generated papers were each matched with a human-written paper on the same problem, with 0.5 as chance. SciSlop v2 is being written with community contributions, and users can submit a link or upload a PDF or LaTeX .zip, receiving a private report key; uploaded files are deleted after analysis while only the report and rendered figures remain.", "body_md": "# Science, or *slop*?\n\nA paper can read well sentence by sentence while its science does not connect. Unlike token-level AI detectors, the Science Slop Index reads the *scientific reasoning*: whether sections build on one another, claims and citations are argued, and the method and evidence can be inspected.\n\nPaste a link or upload a PDF / LaTeX .zip. Your report is private: you get a key, and you decide on the report whether to list it.\n\n**SciSlop v2 is being written with the community.** Flag slop on any paper and become a co-author. How ↓\n## Papers analyzed so far\n\n · [Leaderboard](https://yerimoh.github.io/leaderboard) · [Gallery](https://yerimoh.github.io/gallery)\n\nClick the center paper to open its full report; click a side one to bring it forward. Highlights on each first page are the findings.\n\n## Team\n\nThe people behind *Science or Slop?* and this site.\n\nHow it works\n\n# Six measures, three planes, one number\n\nEach measure counts the share of a paper's units that show one pattern. The index averages the measures within each plane, then averages the planes, and reports the result out of 100. Higher means more of the patterns that set AI-generated papers apart from matched human papers.\n\n## Reads the paper, not the prose\n\nAI text detectors score sentences: how predictable each token is, how a model would have phrased it. A paper written or revised by a language model can pass that test and still fail as science. Each paragraph looks fine on its own while the connections between paragraphs do not hold: a figure that no section ever refers to, a claim that arrives before anything leads up to it, a related-work paragraph that lists citations without relating them, a method diagram crowded with hyperparameters, a results table with no concrete example anywhere in the paper.\n\nThe Science Slop Index counts these breakdowns. It reads the whole paper, groups six patterns into three planes, and asks three questions. Below, each plane is illustrated with real findings from the papers analyzed on this site; use the arrows to browse them, and click a finding to open it on the paper.\n\n## What happens to a paper\n\nEvery report receives a key that reopens it later. Uploaded files are deleted after analysis; only the report and rendered figures stay.\n\n### 1Read\n\nLaTeX source is read directly; arXiv links fetch the source and the PDF. PDFs are rebuilt into sections, captions, references, and citations.\n\n### 2Measure\n\nFour measures are exact counting rules. Argument graph and Figure exposition ask a language model to label sentences and read the method figure.\n\n### 3Highlight\n\nEvery flagged unit is placed back on the PDF, colored by plane, with a graph per measure. Export as highlighted PDF, JSON, or CSV.\n\n## The six measures\n\nAfter Table 1 of the paper. Each score is a share of units: the numerator over the denominator. Pair accuracy is how often that measure alone ranks the human-written paper above its AI-generated counterpart on SciSlopBench.\n\n## One number\n\nThe index averages the measures within each plane, then averages the three planes. A measure that does not apply to a paper is skipped, not counted as zero.\n\nThe four bands are descriptive cut-points on the share of flagged units, not a probability of AI authorship.\n\n## Reported pair accuracy\n\nTable 2 of the paper: on SciSlopBench, 390 AI-generated papers each matched with a human-written paper on the same problem. 0.5 is chance.\n\n# Leaderboard [↗](https://yerimoh.github.io/how)\n\nPapers analyzed on this site, ranked by Science Slop Index. Higher means more scientific slop. Click a row to see its six measures; open the report for every finding on the paper.\n\n[+ Add a paper](https://yerimoh.github.io/)\n\nGallery\n\n# Papers analyzed so far\n\nPapers analyzed from public links, and uploads whose owners chose to list them. Ranked by Science Slop Index.\n\nView report\n\n# Open a report\n\nEnter the report key you received when you submitted your paper.\n\nPropose a pattern\n\n# Propose a scientific slop pattern\n\nSeen a recurring way AI-generated papers break as science that the six measures miss? Name it and say what a reader notices. We do the rest: implement it, run it on SciSlopBench, and adopt it if it separates AI from human papers, with your name on it.", "url": "https://wpnews.pro/news/science-or-slop", "canonical_source": "https://yerimoh.github.io/scientific-slop-demo/", "published_at": "2026-10-02 07:36:18+00:00", "updated_at": "2026-10-02 08:07:40.386332+00:00", "lang": "en", "topics": ["ai-safety", "ai-research", "artificial-intelligence"], "entities": ["Science or Slop?", "Science Slop Index", "SciSlopBench", "SciSlop v2", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/science-or-slop", "markdown": "https://wpnews.pro/news/science-or-slop.md", "text": "https://wpnews.pro/news/science-or-slop.txt", "jsonld": "https://wpnews.pro/news/science-or-slop.jsonld"}}