RENDER benchmark shows memory formats boost LLM accuracy up to 72.6 points

A new benchmark called RENDER shows that memory formats can boost large language model accuracy by up to 72.6 points, with matched-budget 'resolved packets' outperforming recency-truncated raw dialogue by 42.4–72.6 points on 500 LongMemEval questions across nine models. The study, posted on arXiv, found that deployed-style memory templates varied by 24.6–48.8 points within the same model, indicating that the format of memory inputs is a critical factor in production RAG and memory systems.

arXiv https://arxiv.org/abs/2608.23568 RENDER benchmark shows memory formats boost LLM accuracy up to 72.6 points Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated. Matched-budget “resolved packets” beat recency-truncated raw dialogue by 42.4–72.6 points on 500 LongMemEval questions across nine models, and deployed-style memory templates varied by 24.6–48.8 points within the same model. For production RAG/memory systems, the format you feed the model—summary, typed record, ChatGPT-style memory entry, or raw transcript—is not a neutral implementation detail; it can dominate measured quality, so evaluations must lock or report the reader-facing artifact before comparing retrievers, memory stores, or models. LLMs perform significantly better when memory inputs are structured e.g., ChatGPT-style entries rather than raw dialogue, with matched-budget resolved packets outperforming raw dialogue by 42.4-72.6 points. This highlights the critical importance of memory/RAG system design, as the way information is presented to the model can dramatically impact accuracy, making it essential to optimize the reader-facing artifact for production use.