Zhening Li, Omar Khattab, Armando Solar-Lezama and colleagues at MIT CSAIL build JAZ, an agent framework whose only primitive is an LLM-backed invoke function, and test whether it can replace dedicated memory and self-improvement systems.
Ask this paper #
Single primitive. invoke lets the LLM write arbitrary executable code, including recursive calls to invoke, and every input and all interaction history are variables in the code environment.
Language view. The authors treat invoke as a function whose implementation an LLM supplies at runtime on each call, generalizing existing code-mode agent loops.
Hooks, not subsystems. Programmers add constraints and monitoring through built-in hooks instead of attaching memory stores, file systems or custom tools.
Memory result. With prompting only, JAZ beats Letta (MemGPT) by 8% at half the cost on the recall-heavy part of StuLife, where recall exceeds the context window.
Self-improvement result. On AppWorld it beats ACE by 4% at lower cost.
Abstract #
Modern language-model agents are built around the \textit{agent loop}, where the LLM is placed in an environment exposing a set of tools, and the LLM has full control over the workflow by alternating between tool calls and observing their output. However, certain workflows currently require additional engineering beyond the agent loop itself, such as memory systems and self-improving systems. We built an LLM agent framework, JAZ, to explore the extent to which a minimal harness that is little more than the agent loop itself can accomplish tasks these specialized systems are built for. JAZ exposes a single LLM-based primitive invoke and provides a set of built-in hooks that allow the programmer to apply constraints and monitoring. Generalizing existing code-mode agent loops, \texttt{invoke} is the simplest loop that satisfies two defining properties: (1) the LLM can write arbitrary executable code that can include recursive \texttt{invoke}; (2) everything visible to the LLM --- all inputs to \texttt{invoke} as well as its interaction history with the code environment --- are variables in the code environment. We motivate our design from first principles, viewing \texttt{invoke} as a language primitive representing a function whose implementation is provided at runtime by an LLM every time it is called. To validate the design of our core \texttt{invoke} primitive, we evaluate \texttt{invoke} --- with only prompting, no manually designed tools, harness, or external systems (e.g., memory or the file system) --- on workflows traditionally implemented through specialized external harnesses. On long-horizon workflows requiring recall beyond the context window, JAZ invoke outperforms Letta (MemGPT) by 8% at half its cost on the recall-heavy portion of StuLife. On continual self-improvement, JAZ invoke outperforms ACE by 4% at a lower cost on AppWorld.