Skills for Real Scientists – Steer Away AI Slop A new set of agent skills called "Skills for Real Scientists," built on Matt Pocock's "skills for real engineers," lets coding agents run reproducible scientific experiments one "probe" at a time, recording the expected result before each run and choosing the next probe from the evidence. The science flow replaces the engineering flow's up-front spec-and-tickets plan with a chain of probes, each leaving a Git-tracked lab notebook and rerunnable files in a `.science/` folder, with results reaching the main codebase only through a `/promote` step. The author argues that steering is what separates real findings from "AI slop," since an unsteered language model returns the most common answer, the middle of everything it has read. Built on Matt Pocock's skills for real engineers https://github.com/mattpocock/skills . Agent skills for reproducible scientific experiments with any coding agent. Agree on one experiment at a time, record the expected result before running it, and use the evidence to choose what to run next. Every run leaves a Git-tracked lab notebook and rerunnable files. Matt's engineering flow plans the whole route first: /to-spec writes down the goal, /to-tickets splits it into tickets, and /implement-spec builds them, many in parallel. That works because software is predictable once you agree on what to build. Research and scientific exploration aren't. A plan is a map, and in research nobody has mapped the territory yet. You learn what's relevant by running something and looking at the result: each result shows you more of the territory, and it often shows that the route you planned goes nowhere. So the science flow plans one experiment, called a probe, at a time, and you choose each next probe after seeing the last result. The study's notebook is the map of your scientific question, redrawn after every probe. Ask a language model a question without steering it, and it gives you the most common answer, the middle of everything it has read. New findings sit away from that middle, and only a clear intention moves the work away from it, away from what you could call AI slop. So before each run the agent writes down the standard answer, and the result shows how far from it reality lands. That distance is what makes the work worth doing, and a rough measure of how unique it is. | | Engineering flow | Science flow | |---|---|---| | The unknown | How to build it | What's relevant, and so what to do next | | Plan | Every ticket, up front | The first probe and a list of ideas; each next probe chosen when it's next | | Shape of the work | A task graph, run in parallel | A chain of probes, each one checked by you | | Where work happens | Feature branches, into the codebase | Scratch probes in a study folder; the main tree changes only by promotion | | The starting doc | A spec, which is a contract | A direction, which is expected to change | | Failure | A bug to fix | Unexpected results teach; invalid runs need fixing | When a protocol stops changing, it's predictable again, and Matt's engineering flow takes over. Everything the science skills write goes in a .science/ folder at the root of your repo: .science/ SETUP.md your setup library/ what you read: papers, software docs; one page per question, the raw search results data/ your input data or data/ at the repo root, your choice