There’s more and more chatter about RSI, or Recursive Self Improvement, as we get closer to the Singularity. Google, along with researchers from Google DeepMind, the University of Maryland, and the University of Virginia, have published a paper describing Dream-RSI, a system that lets an AI agent get better at exploring problems over time — without ever retraining or fine-tuning the underlying model.
The framework tackles a specific bottleneck in how AI agents are increasingly being used for scientific and algorithmic discovery. Tools like AlphaEvolve have shown that coding agents can propose, test, and refine candidate solutions across thousands of cycles to land on new algorithms or mathematical results. But as these search spaces get larger, the strategy an agent uses to explore them — which ideas to pursue, which to abandon, how much to run in parallel — becomes just as important as the agent’s raw capability. Get the exploration strategy wrong, and an agent can burn enormous amounts of compute chasing dead ends.
The obvious fix would be to let the agent learn a better exploration strategy as it goes. The problem, according to the researchers, is that doing this “online” is brutally expensive. You can’t tell if a new exploration policy is good until you’ve run it for a long time and seen where it leads, and there are countless possible policies to try. Every test requires a fresh, costly round of real agent calls and evaluations.
Dream-RSI’s workaround is to stop treating past discovery attempts as disposable. Instead, it converts the full record of a completed search — every branch tried, every outcome scored — into what the paper calls a “replay simulator,” or a world. Since every outcome along every path is already known, a new exploration policy can be tested by simply replaying it against that historical record instead of running the actual agent again. There’s no new code being generated, no new evaluation being run — just a lookup against outcomes that already happened.
The researchers describe this as the agent “dreaming”: it imagines thousands of alternative versions of its own search strategy, plays each one out against its memory of past attempts, and keeps whichever version scores best on quality, cost, and how well it uses parallel compute. That improved policy then gets deployed for a real round of exploration, which produces a fresh discovery history that expands the simulator pool for the next round of dreaming. It’s a loop that borrows directly from model-based reinforcement learning approaches like DeepMind’s own Dreamer line of world models, applied here at the level of search strategy rather than raw perception and control.
Crucially, the coding agent itself, the evaluator, and the model doing the actual work — Gemini-3.1 Pro and Gemini-3.7-Flash in the paper’s experiments, run via the Gemini CLI — never change. Only the exploration-policy code, essentially a lightweight orchestration layer sitting on top of the agent, gets rewritten between rounds. This is the detail that makes the framing of “recursive self-improvement” here different from the kind Google DeepMind researchers have talked about previously in the context of one model generation training the next. Dream-RSI’s recursion happens entirely at the strategy layer, not the weights layer.
The team tested the approach across three fairly different domains. In algorithm engineering, they had it discover implementations of the Lasso regularization path, a widely used technique in statistics — echoing the kind of algorithm-discovery work AlphaEvolve became known for after cracking Ramsey number bounds and improving matrix multiplication bounds earlier this year. Here, Dream-RSI produced solvers that beat standard libraries like scikit-learn and glmnet on six held-out datasets, while using roughly 162 times fewer agent calls than a prior discovery system called SimpleTES, and about 1.7 times fewer than a version of Dream-RSI’s own setup with a fixed, non-improving exploration policy.
On mathematical optimization tasks — sum-difference, autocorrelation inequalities, and circle packing — Dream-RSI matched or beat strong existing discovery systems while using a fraction of the generations SimpleTES needed, translating to more than 50 times lower compute cost in some cases. And on GPU kernel engineering, tested on KernelBench workloads including VGG16 and LayerNorm, it reached comparable performance using up to 2.43 times fewer generations, or delivered up to roughly double the performance at the same compute budget, compared to keeping the exploration strategy fixed.
One finding the researchers flag as counterintuitive: they also tried a simpler way of using history, where past search trajectories were summarized into plain-language advice and stuffed into the agent’s prompt. That approach consistently underperformed letting the agent replay and test strategies against the full simulator, suggesting that overly specific guidance about where to search next tends to narrow exploration too aggressively rather than help it.