Ao Qu and collaborators open-source Reef for learning from live agent interactions

MIT researcher Ao Qu and a distributed group of AI researchers open-sourced Reef on September 15th, an Apache-2.0 stack that turns inference logs into a versioned learning pipeline for both model weights and agent software. Reef records inference traces and feedback through a standard OpenAI-format endpoint, returns an x-reef-agent-record-id receipt per interaction, and evaluates candidate updates before publication, leaving the serving version unchanged when a candidate is rejected. The project site identifies Slime and SGLang in Reef's model-weights path and lists recipes for personalized-chat learning, scientific discovery, prompt evolution and skill development.

Ao Qu and collaborators open-source Reef for learning from live agent interactions The MIT researcher and a distributed contributor group built an Apache-2.0 stack that updates model weights and agent harnesses from live experience. By Ryan Merket https://runtimewire.com/author/ryan-merket · Published Primary source: Hugging Face Newsroom https://huggingface.co/blog/quao627/your-inference-server-is-secretly-a-learner-reef Why it matters Reef turns inference logs into a versioned learning pipeline for both model weights and agent software. Its release controls evaluate candidate updates before publication and leave the serving version unchanged when a candidate is rejected. MIT researcher Ao Qu and a distributed group of AI researchers open-sourced Reef on September 15th, putting agent learning directly inside the infrastructure that serves user requests. The project describes Reef as fully open source. In a Hugging Face community article https://huggingface.co/blog/quao627/your-inference-server-is-secretly-a-learner-reef?ref=runtimewire , Qu and his collaborators argue that the standard model lifecycle - train, evaluate, deploy, serve - breaks down when agents are expected to learn from their work. Reef records inference traces and feedback, feeds those records into learning jobs, evaluates proposed changes, and publishes accepted model or agent versions without stopping the serving system. Qu has been working toward that loop through research on agent memory, planning and experience-based learning. He is a Ph.D. student at MIT's Institute for Data, Systems, and Society, after graduating from Vanderbilt University with degrees in mathematics and computer science, according to his MIT profile https://m3s.mit.edu/our-team/ao-qu?ref=runtimewire . Reef extends that research into a usable systems layer. Qu's central bet is that an inference server should collect learning material rather than discard each interaction after returning an answer. Every response gets a receipt A request reaches Reef through a standard OpenAI-format endpoint. Reef returns the model response along with an x-reef-agent-record-id , effectively giving the interaction a receipt. An application can later attach a reward, evaluator result or user correction to that record through Reef's feedback endpoint. Reef stores those requests, execution traces and feedback as a structured experience stream. Training processors determine which records are eligible for learning, and evaluation modules decide whether resulting candidates should be published. Serving continues while the learning work runs asynchronously. The open-source repository https://github.com/Human-Agent-Society/reef?ref=runtimewire divides that cycle into four stages: serve, observe, grow and commit. The first two collect interactions and match feedback to them. The third runs a configured learning recipe. The final stage evaluates the resulting artifact and decides whether it should replace the version already serving traffic. That release gate carries much of Reef's practical value. Continual learning can degrade a working system as easily as it can improve one. According to the project documentation https://github.com/Human-Agent-Society/reef?ref=runtimewire , Reef versions model and harness artifacts, publishes approved candidates as new releases, and leaves the current serving version unchanged when a candidate is rejected. The project site https://www.reefinfra.ai/?ref=runtimewire identifies Slime and SGLang in Reef's model-weights path and lists recipes for personalized-chat learning, scientific discovery, prompt evolution and skill development. The agent harness is part of the product Reef's broader contribution is its treatment of the agent harness as software that can be updated from experience. That distinction matters because many failures blamed on a model originate in the surrounding application. A tool may receive the wrong arguments. Memory may retrieve irrelevant context. A prompt may invite a plausible answer where the application requires a verified one. Updating weights for every such problem is expensive and often unnecessary. Reef uses a harness-evolution backend called Cordis to analyze trajectories and propose changes to the surrounding agent code. An accepted harness is distributed as a new installable version. Qu is working with researchers whose prior projects cover both sides of this problem. According to Bo Liu's personal site https://benjamin-eecs.github.io/?ref=runtimewire , he is a University of Washington visiting researcher and previously worked at Meta FAIR on scalable language-model self-improvement and self-play. Han Zheng's research profile https://mikezheng777.github.io/?ref=runtimewire identifies him as an MIT Ph.D. student whose work spans reinforcement learning, combinatorial planning and self-evolving agents; he also worked on agentic post-training at Amazon AGI. Zijian Zhou's research profile https://www.franciszzj.com/?ref=runtimewire identifies him as a Meta research scientist who previously worked at ByteDance and Horizon Robotics before completing his doctorate at King's College London. Simon Yu's public GitHub materials https://github.com/simonucl?ref=runtimewire describe a runtime substrate for reversible, Git-like agent execution traces that support forking and replaying past states. The contributor list also includes Xuan Jiang, Wenhao Chai, Zoe Dusy and Xinkai Zou. The group looks closer to an open research collective than a conventional startup. Reef is maintained under the Human-Agent-Society GitHub organization and released under the Apache-2.0 license. The public materials establish neither a commercial product nor a corporate structure. That gives outside developers broad freedom to test Reef while leaving its long-term maintenance model open. Agent infrastructure is moving beyond tokens Reef arrives as researchers and infrastructure vendors confront the operational differences between chat completion APIs and long-running agents. Agents carry persistent context, call tools, wait on external systems and produce trajectories that are more complicated than a stream of generated tokens. A paper on agent-native serving https://arxiv.org/abs/2607.29069?ref=runtimewire examines the infrastructure requirements of agentic serving systems. Reef addresses an adjacent problem: preserving experience from live serving so it can support later improvements. Existing projects cover pieces of that workflow. vLLM and SGLang focus on inference, while Slime, veRL and AReaL concentrate on reinforcement-learning workloads. Reef's claim is that live serving, feedback collection, training, harness editing, evaluation and version delivery belong in one control loop. The evidence available at launch consists of the code, architecture and project-run demonstrations. The Hugging Face article shows personalized-chat and optimization examples, though it does not establish how Reef performs under production traffic, noisy human feedback or adversarial attempts to influence future versions. Those conditions will determine whether continual learning is useful outside controlled tasks. Feedback can encode user preferences, factual corrections and successful tool executions. It can also contain private information, manipulation attempts or rewards that encourage shortcuts. Reef's evaluation and versioning controls provide a place to enforce safeguards, but each deployment still needs its own tests, approval policy and data-handling rules. Qu and his collaborators have chosen a practical entry point for a field crowded with claims about recursive self-improvement. Reef does not require an agent to rewrite its entire intelligence stack. It gives developers a server that remembers what happened, proposes bounded changes and keeps a version history when those changes ship. That is a smaller promise than an autonomous system endlessly rebuilding itself. The public repository https://github.com/Human-Agent-Society/reef?ref=runtimewire provides installation, quickstart and run instructions.