At the end of May, I attended the SciFM26 conference hosted at UChicago which had many speakers discussing agentic science and the future of AI+Science. This piece will attempt to capture the most interesting discussions I had about the future of AI+Science.
Suppose you have a cutting edge autonomous lab that can plate and incubate bacteria, generate arbitrarily engineered plasmids, transform the plasmids into the bacteria, and run a slew of analytics on the cultures without any human intervention. You’re performing a study on the ability of bacteria to use plasmids to mass-produce some biochemical, and you’ve set up the system to do reinforcement learning on the genetic code of the plasmid to optimize the yield of your product. The system works well, and at the end of your months-long training run, you have a new high-yield recipe ready to scale up in a bioreactor.
However, unbeknownst to you, one of the failed plasmids coded for a powerful new antibiotic which promptly killed all of the bacteria and showed up as 0% yield in the analysis. Your algorithm rightfully scrapped this design, and the policy learned to avoid this region of sequence space.
Unlike in the serendipitous discovery of the penicillin-producing fungus during unrelated research by Alexander Fleming, the antibiotic here goes unnoticed. My point here is not to bash autonomous labs; my point is that they come with two implicit assumptions which appear to conflict with how science has historically advanced:
In the penicillin example, what kind of system would be able to note that in addition to 0% yield, the biomass had gone to zero, suggesting that the bacteria had died? How would you distinguish an antibiotic-like effect from a fluke death of all the bacteria due to a system malfunction? Would you have to have an LLM interpreting all available data on any given run to detect anomalies such as this? Any autonomous system must have some kind of metric it is trying to maximize, and obviously the target yield is what you nominally want, but it’s not quite what an inquisitive scientist like Fleming would be aiming for such as an abstract “some kind of useful discovery”. Goodhart’s Law strikes again.
One system that might be able to systematically capture “serendipitous discovery” is curiosity-driven learning where the system is both constantly predicting the outcome of an experiment and comparing the outcome to its prediction. When the prediction is sufficiently wrong, the model is “intrinsically motivated” by curiosity to squash this prediction error without any hard reward like yield pushing it to do so. In such a system, the death of all bacteria would be an unexpected result, and it would be pushed to explore its root cause.
Perhaps more interestingly, what if the accidental byproduct is actually some hyperspectral reporter far superior to anything that currently exists, but your autonomous lab system doesn’t have the right hyperspectral camera to capture the remarkable result? It’s impossible to quantify how many of these kinds of tangential results slip through human-run labs without notice, but accidental discovery is something one might expect to become more common with ultra high-throughput labs and hope a good AI scientist would be able to catch and publish.
The “convex hull” assumption of autonomous labs is likely the more important of the two assumptions because there is strong evidence many scientific discoveries stem from the development of entirely new tools, as Charles Yang also points out. Any bet on autonomous labs is a claim the solution lies in some combination of the tools you’ve linked together, so it’s critical you have the right tools at your disposal. In other words, building an autonomous lab is analogous to initiating a fine-grained search in the area under 10 different streetlights, but it’s also plausible you need a new streetlight to locate your target (i.e. inventing a new device).
Freeman Dyson once described these tool-driven revolutions of science as being even more significant than Kuhn’s “paradigm shifts” in The Structure of Scientific Revolutions: “In the last five hundred years we have had six major concept-driven revolutions, associated with the names of Copernicus, Newton, Darwin, Maxwell, Einstein and Freud, besides the quantum-mechanical revolution that Kuhn took as his model. During the same period there have been about twenty tool-driven revolutions, not so impressive to the general public but of equal importance to the progress of science.”
Consider the stated goal of Periodic Labs - they want to make useful high temperature superconductors. One of their advisors is ZX Shen, the GOAT of exotic superconductivity. He certainly has many ideas and recipes to try, and they will build fully autonomous systems with state-of-the-art molecular beam epitaxy chambers with RHEED, ARPES, RIXS, XPS, STM, and other sensors. But what if the tool you need to make room temperature superconductivity is a custom-built thin film manipulator which must operate in the ultra-high vacuum?
If you were to enumerate the tools a researcher at a major university has available to them, it would be impossible to capture them all. Andre Geim got the Nobel Prize for discovering graphene with Scotch tape. Because of this obscenely long tail of tools needed, I’m not so sure a complete AI scientist is possible until a human-level dexterous robotic system is able to construct custom tools using general reasoning in the way a human does. Good science is fundamentally about doing what has not been done before, so the convex-hull assumption appears weaker and weaker under scrutiny. In this light, autonomous labs look like fancy optimizer loops, not scientists per se. Lila will almost certainly be able to optimize the yield of the various factories it applies its agents and labs to, but one might argue this is more engineering than science.
This is not to say autonomous labs are not incredibly useful and inevitably at the core of future AI scientists, but significant thought must also go into what it means to emulate a scientist with an algorithm or agent. For all the talk about and fundraising for autonomous lab setups and all the tools they will incorporate (Lila, Periodic, Dunia, Radical, etc), I’ve seen very little directly addressing what the brain will look like apart from just “make an LLM run the autonomous lab using the scientific method”.
General intelligence via frontier reasoning models will certainly play a part in hypothesis generation, data analysis, and execution, but this brings us to a paradox regarding scientific hypothesis generation: many in tech will say that with AI, “ideas are cheap, execution is everything”, but also that “a good idea is worth a million experiments.” The resolution to this paradox is just that “good ideas are still the bottleneck” (something akin to taste), but now we’ve just gone back to the central question of metascience which is “how do we come up with good hypotheses and test them rigorously”. Having taste exceeding the best scientists is a property one might use to define superintelligence, so perhaps the answer will be to just wait until that happens and we’ve built creative and dexterous AI scientists.