Large language models have enormous context windows now. That does not mean they use all of that context reliably.
As prompts grow, models can miss details, lose track of relationships, or produce plausible summaries instead of doing the exhaustive work a question requires. The Recursive Language Models (RLM) paper proposes a different interface: keep the large context outside the model, expose it as a variable in a persistent programming environment, and let the model inspect, partition, and recursively query smaller pieces.
We rebuilt that method with an unusual constraint:
OPENAI_API_KEY
;gpt-5.4-mini
for both the RLM root and every subcall;The result was encouraging, expensive, and more nuanced than “cheap model equals frontier model.”
A normal model call looks roughly like this:
large prompt -> model -> answer
An RLM instead gives the root model metadata about the input and a Python REPL containing the real context:
question
|
root model
|
persistent REPL holding the context
|-- inspect and search with code
|-- split context into useful chunks
|-- call smaller LMs over those chunks
|-- validate and aggregate results
`-- return the final answer
The important detail is that the root model does not need to carry every document, record, tool result, and partial answer in its own context window. Large intermediate values can remain in REPL variables. Subcalls receive focused, locally understandable tasks.
That makes RLM less like a bigger prompt and more like an out-of-core data-processing system whose semantic operator happens to be a language model.
We used an OOLONG trec_coarse
validation example from the protocol described in the RLM work.
The input was a 308,367-character context containing 3,182 general-knowledge questions. Each question implicitly belonged to one of six answer types:
The labels were not present in the context. The task was to infer the labels and identify the least-common category.
We compared:
gpt-5.6-sol
Codex call.gpt-5.4-mini
.The direct frontier call answered abbreviation
and scored zero. The mini-only RLM answered numeric value
, matching the gold answer.
| Method | Result | Model calls | Elapsed time |
|---|---|---|---|
| Direct frontier call | Incorrect | 1 | 40.1 seconds |
RLM with gpt-5.4-mini only |
|||
| Correct | At least 238 | 6,120.3 seconds |
The RLM root first inspected the structure of the context. It then classified chunks, retried malformed responses, reduced the chunk size, reclassified all 3,182 questions using structured JSON outputs, checked that it had coverage, and calculated the minimum.
This is exactly the sort of work that a direct model call often approximates but a recursive program can force itself to perform.
Getting the final answer right did not mean every intermediate judgment was right.
We compared the mini model's inferred counts against the validated labels:
| Label | True count | Mini inferred |
|---|---|---|
| Numeric value | 398 | 402 |
| Entity | 521 | 623 |
| Human being | 544 | 488 |
| Location | 571 | 493 |
| Abbreviation | 571 | 560 |
| Description and abstract concept | 577 | 616 |
The model made substantial row-level classification errors. It still found the correct minimum because numeric value had a 123-item margin over the next-smallest true category.
That distinction matters. This run shows that decomposition changed the outcome and allowed a cheap model to solve one problem that the direct frontier call missed. It does not prove that the cheap model reconstructed the data exactly, and one row does not establish general equality between the two systems.
The honest claim is narrower:
On suitable long-context tasks, a cheap model inside an RLM can match or outperform a direct frontier-model call.
The paper evaluates four useful task shapes:
OOLONG requires labeling and aggregating information spread throughout a large input. Real applications include:
Our experiment belongs to this category.
BrowseComp-Plus requires joining evidence across documents in a very large offline corpus. Analogous applications include:
The paper includes LongBench-v2 CodeQA, where questions require reasoning across files in a codebase. Probable uses include:
OOLONG-Pairs asks the system to construct relationships between combinations of records. Applications could include:
These workloads can grow quadratically, so they need strict budgets and deterministic post-processing.
While exploring our local Claude Code history, we found a single session transcript that was 242 MB and contained 39,570 JSONL records. All project transcripts together occupied about 3.6 GB.
The large session was not 242 MB of useful conversation:
This is an excellent RLM-shaped problem.
A deterministic first pass can stream the JSONL, hash duplicate attachments, reconstruct parent-child event relationships, merge subagent logs, and extract messages, commands, file changes, tests, commits, errors, and outcomes. An RLM can then analyze normalized episodes and recursively build:
The final report should cite session IDs, event IDs, timestamps, commands, and Git commits. Otherwise, it is merely another plausible summary.
The same decomposition pattern should transfer to:
The recurring requirement is not simply “the input is long.” A good RLM task has four properties:
RLM is a poor default for:
Our successful row took roughly 102 minutes. That is acceptable for a research run or an overnight audit, not for an interactive endpoint.
The useful abstraction is not an OOLONG runner and not one universal prompt. It is a context-compute runtime with a small set of reusable recipes:
run(
context,
objective,
recipe,
answer_schema,
verifier,
budget
) -> answer + evidence + validation + trajectory + usage
Initial recipes could include:
aggregate_records
evidence_synthesis
repository_analysis
cross_record_join
timeline
candidate_ranking
For our intended configuration, the Codex backend would keep both root and subcalls locked to gpt-5.4-mini
. A frontier model would appear only in evaluation runs, never inside the RLM call tree.
Production use would also require an isolated execution environment, call and token limits, schema validation, redaction, prompt-injection defenses, resumable runs, and source-level evidence for every important claim.
The one-row result is a proof of mechanism, not a benchmark victory.
The immediate research questions are:
RLMs do not magically turn a cheap model into a frontier model. They change the computation available to that model. Sometimes that difference is enough to turn a wrong one-shot answer into a correct, auditable process.
That is a more interesting result than the slogan.