{"slug": "stop-your-llms-from-forgetting-how-a-2016-string-algorithm-solves-ai-s-biggest", "title": "Stop Your LLMs from Forgetting: How a 2016 String Algorithm Solves AI's Biggest Memory Loss Problem", "summary": "A developer adapted a 2016 string concatenation algorithm, called the Pyramid Method, to solve memory loss in large language models (LLMs) when aggregating multiple documents. The new approach, Pyramid Aggregation, organizes document merging into a balanced tree topology, achieving a 95% speedup and mitigating positional biases. The work was published as a preprint on Zenodo.", "body_md": "Have you ever tried to read a massive pile of reports and summarize them in under 50 words? It’s hard. Now, imagine asking a cutting-edge Large Language Model (LLM)—like Gemini—to do it.\n\nYou might think AIs have perfect memories, but they don't. When forced to aggregate information from dozens of documents under strict length constraints, AIs suffer from severe \"memory loss\" biases. They either ignore the middle of your documents or completely forget the older information they read first.\n\nIn this article, we’ll introduce a simple yet powerful solution called **Pyramid Aggregation**. Intriguingly, this method is adapted from a **10-year-old string concatenation algorithm** that was originally designed to make basic programming languages run faster. By applying it to modern AI, we solved the forgetting problem and achieved a **95% speedup** in processing time.\n\nLet's dive in!\n\nOn October 13, 2016, I published a technical post on my blog titled [\"Improved Algorithms for Summation of Array Elements\"](https://tanaikech.github.io/2016/10/13/improved-algorithms-for-summation-of-array-elements/). Recently, I officially archived this work on [Zenodo](https://zenodo.org/records/21232389).\n\nBack in 2016, the goal of that paper was simple: find a highly efficient way to join (concatenate) thousands of text strings together. In the older version of Google Apps Script (before the V8 engine), standard sequential text joining was incredibly slow and memory-intensive because the computer had to rebuild the text footprint over and over again ($O(N^2)$ complexity). By proposing a hierarchical tree-like method—which I called the **Pyramid Method**—we restricted the active memory growth to a linear scale ($O(N)$), resulting in a massive **99.7% reduction in execution costs**.\n\nWhile review-proofing that paper for Zenodo recently, a spark went off in my head:\n\n\"Could this exact same pyramid tree algorithm be used to optimize how today's Generative AIs aggregate multiple documents?\"\n\nWhen an LLM processes long texts, the computer's attention calculations scale quadratically ($O(L^2)$) relative to the input length—just like the old string joining problem!\n\nTo test this theory, I designed an experiment comparing three text aggregation workflows: **Batch-Concatenate**, **Sequential Update**, and our proposed **Pyramid Aggregation**. The results were published as a preprint on Zenodo: [\"Pyramid Aggregator: Mitigating Information Loss in Multi-Document Fact Extraction via Hierarchical Merging\"](https://zenodo.org/records/21252820).\n\nTo understand why we need a pyramid, we first need to look at how AIs behave when you feed them a lot of documents under a strict word limit (e.g., summarizing 32 system logs in under 50 words without just listing names).\n\nTraditionally, developers use one of two methods, both of which suffer from severe positional biases:\n\nInstead of processing everything at once or step-by-step, **Pyramid Aggregation** organizes document merging into a balanced tree topology.\n\nHere is how the three workflows compare visually:\n\n**Figure 1** displays the structural difference between the three approaches:\n\nIn a basic implementation, running 11 separate LLM calls (8 for Level 1, 2 for Level 2, and 1 for Level 3) would be painfully slow if executed one by one. This is where the **Antigravity Python SDK's concurrency control** comes into play.\n\nIn the Pyramid workflow, the 8 nodes in Level 1 are completely independent of each other. The Antigravity SDK leverages event-driven asynchronous execution to process all 8 Level-1 calls **at the exact same time (in parallel)**. Once those return, the SDK immediately triggers the 2 Level-2 merge calls in parallel.\n\nUsing the SDK's async context (`async with Agent(config)`\n\n) combined with `asyncio.gather`\n\n, we can orchestrate this entire multi-agent hierarchy concurrently. The SDK handles connection pooling and rate limits automatically behind the scenes, allowing us to transition from a linear time complexity ($O(\\theta)$ rounds) to a highly efficient logarithmic scale ($O(\\log_{\\phi} \\theta)$ rounds). Every document maintains an identical 3-step depth to the top, completely eliminating position bias while ensuring near-instant execution.\n\nWe put these three methods to the test using 32 synthetic system logs (each reporting an error in a specific software module) and a lightweight LLM (`gemini-3.1-flash-lite`\n\n) orchestrated via the Antigravity Python SDK.\n\nWe set a strict limit: the final summary had to be under 50 words and could not list more than 3 module names in a single sentence (preventing the AI from cheating by just listing everything).\n\nHere are the actual results from our benchmark:\n\nLooking at **Figure 2**, we can draw three critical conclusions:\n\nIn **Panel A**, we tracked which documents (from 0 to 31) made it into the final summary:\n\nUnder a strict 50-word limit, it is mathematically impossible to list all 32 names. So what does the AI prioritize? In **Panel B**, each method displays two bars: a **blue bar** for **Macro Domain Coverage (%)** and an **orange bar** for **Micro Facts Counted (%)**:\n\nWhy is a **0.0% Micro Recovery** actually a victory for the Pyramid method? Think of it this way:\n\nHere is a visual representation of this analogy:\n\nAs shown in **Figure 3**, the baseline approach (left) only keeps a few active students (like Sarah, Mike, and Ben) while leaving the rest of the class greyed out and forgotten. On the other hand, the Pyramid approach (right) synthesizes the entire classroom by dividing all 32 students into functional categories (Athletes, Artists, and Programmers). In other words, the 0% in Panel A/B isn't a failure to remember—it is a proof of **high-level semantic synthesis**. The AI successfully summarized the data rather than lazily cropping it.\n\nPerhaps the most dramatic result is the execution time, shown in **Panel C** (where lower is better):\n\nBy taking a simple tree-reduction algorithm written in 2016 for basic string concatenations and mapping it to the attention constraints of modern Large Language Models, we solved a fundamental issue in AI document synthesis.\n\nPyramid Aggregation acts as a hierarchical semantic filter, turning raw, massive text inputs into balanced, high-level summaries without dropping historical context. Thanks to modern asynchronous orchestration SDKs like Antigravity, we can deploy this tree structure at scale, achieving near-instant results.\n\nIf you are building AI agents, RAG pipelines, or log analysis tools, stop throwing all your text into a single prompt or chaining them sequentially. **Build a pyramid instead!**\n\nGoogle Cloud credits are provided for this project.", "url": "https://wpnews.pro/news/stop-your-llms-from-forgetting-how-a-2016-string-algorithm-solves-ai-s-biggest", "canonical_source": "https://dev.to/gde/stop-your-llms-from-forgetting-how-a-2016-string-algorithm-solves-ais-biggest-memory-loss-problem-240f", "published_at": "2026-07-08 06:08:58+00:00", "updated_at": "2026-07-08 06:28:34.811245+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "generative-ai", "developer-tools"], "entities": ["Gemini", "Google Apps Script", "Zenodo", "Antigravity Python SDK", "Pyramid Aggregation", "Pyramid Method"], "alternates": {"html": "https://wpnews.pro/news/stop-your-llms-from-forgetting-how-a-2016-string-algorithm-solves-ai-s-biggest", "markdown": "https://wpnews.pro/news/stop-your-llms-from-forgetting-how-a-2016-string-algorithm-solves-ai-s-biggest.md", "text": "https://wpnews.pro/news/stop-your-llms-from-forgetting-how-a-2016-string-algorithm-solves-ai-s-biggest.txt", "jsonld": "https://wpnews.pro/news/stop-your-llms-from-forgetting-how-a-2016-string-algorithm-solves-ai-s-biggest.jsonld"}}