# Stop Your LLMs from Forgetting (Part 2): How a Graph-Anchor Pyramid Cures AI’s Relational Blindspots

> Source: <https://dev.to/gde/stop-your-llms-from-forgetting-part-2-how-a-graph-anchor-pyramid-cures-ais-relational-blindspots-4m33>
> Published: 2026-07-15 03:22:14+00:00

Have you ever had a brilliant solution get completely crushed by a single comment on a technical blog post?

Just last week, on July 8, 2026, I published a post introducing the [ Pyramid Aggregator](https://medium.com/google-cloud/stop-your-llms-from-forgetting-how-a-2016-string-algorithm-solves-ais-biggest-memory-loss-problem-444cf6b6b24b). It was a beautiful story of technical lineage: we took a string summation algorithm originally designed on October 13, 2016 (to optimize Google Apps Script) and applied it to modern Large Language Models (LLMs). By organizing document merging into a balanced, parallel tree topology, we successfully bypassed the AI's "Lost in the Middle" bias and achieved a

The paper was archived on Zenodo, and the Medium article did great. But then, a reader dropped a comment that pointed out a massive, system-level blindspot.

They wrote:

"Pyramid Aggregator is a great synthesis tool, but it assumes your input set is already perfect. In a real production system, what if your vector search fails to retrieve the root-cause documents in the first place? If the cause is missing, even a bias-free summary is just a bias-free summary of incomplete data."

They were 100% correct.

In real-world operations—like Site Reliability Engineering (SRE) or Cybersecurity—root causes and visible symptoms are often separated by several layers of system logic. If a database tablespace fills up, a flat vector search for "database issues" will retrieve the database logs. But it will completely ignore the low-level voltage drop log from the power grid container, simply because the words "database" and "voltage" are semantically unrelated.

This is the **Multi-Hop Relational Blindspot**.

To solve this, we went back to the drawing board. Today, we are releasing the next evolution of our hierarchical context architecture: **GAP (Graph-Anchor Pyramid)**. By combining **Graph Retrieval (Graph-RAG)**, **Topology-Aware Leaf Grouping**, and **Prompt-Level Semantic Anchoring**, we have created a lossless, parallel framework that recovers **100% of causal chains** in multi-document synthesis tasks—and we've proved it across two generations of Gemini models.

Here is the deep dive into how it works.

When we scale LLM context ingestion using hierarchical summarization, we face a fundamental trade-off: we want high-level semantic summaries, but we cannot afford to lose the critical, fine-grained details (like specific error codes, account IDs, or IP addresses).

Standard hierarchical pipelines break down in two places:

Flat vector databases represent documents as points in a high-dimensional space. While this is great for finding similar documents, it fails when documents are connected by **dependency or causality** rather than similarity.

If a query is `"database connection failure"`

, standard semantic search retrieves:

But it completely misses:

Because Golem and Thor don't mention the word "database", their embeddings are too far away. The RAG pipeline suffers from a search gap, feeding incomplete context to the generator.

Even if we retrieve all the logs, merging them hierarchically causes another issue. Forcing intermediate nodes in the tree to output natural language summaries acts as a **lossy compression (discrete bottleneck)**.

As summaries are merged up the tree, the model's contextual embeddings drift. Under strict word limits (like summarizing 16 system logs in under 50 words), the LLM is forced to prune text. Crucial micro-facts (like specific module names or error codes) are washed out, leaving only generic macro summaries.

In our original Pyramid Aggregator benchmark, when forced into strict length and listing constraints, the **Micro-Fact Recovery Rate dropped to 0.0%**. The AI gave a great high-level summary but forgot the specific entities.

The GAP architecture resolves both issues by adding graph topology and static attention anchors to the hierarchical tree.

Here is the structural comparison between standard RAG, chronological tree aggregation, and GAP:

GAP operates in three unified stages:

Instead of relying on flat vector search, we represent the document corpus as a directed dependency graph $G = (V, E)$. The vertices $V$ are document chunks (logs), and the directed edges $E$ represent logical or causal dependencies (Cause $\rightarrow$ Effect).

When a query is received:

This completely closes the relational blindspot.

Once we have retrieved the documents, we must group them to build the tree. A standard tree-reduction groups documents chronologically or sequentially. However, this mechanical slicing often cuts causal edges (called **Causal Splitting**), grouping the "cause" in Leaf 1 and the "effect" in Leaf 2. Because they are processed separately, their relationship is diluted before they ever meet.

GAP solves this by treating the leaf grouping as a graph partitioning problem. It groups nodes to **minimize edge cuts** on the retrieved subgraph. This guarantees that causally connected logs are grouped together in the same Level 1 leaf prompt, preserving their relationships in the initial, low-level summaries.

To prevent recursive semantic decay, we introduce static **Semantic Anchors** ($A$).

`Golem (ERR-101) -> Mercury (ERR-102)`

).`### SEMANTIC ANCHORS ###`

block within the system prompt of every intermediate and root node.This acts as a "hard link" that bypasses the lossy compression of intermediate summaries, carrying crucial facts all the way to the final root output.

Below is the comparative architectural workflow of the three evaluated pipelines:

We constructed a realistic system incident data set consisting of 16 logs containing 4 independent, multi-hop causal chains (e.g., memory exhaustion leading to database disconnects) mixed with background noise logs:

We evaluated three pipelines:

To make the task extremely challenging and force information loss, we applied three strict constraints:

We ran the benchmarks across two generations of Gemini models: `gemini-3.1-flash-lite`

(lightweight, optimized for low latency) and `gemini-3.5-flash`

(designed for complex reasoning).

Here is the evaluation matrix:

| LLM Model | Pipeline | Causal Recall ($\text{RR}_{\text{causal}}$) | Macro Coverage ($\text{CC}_{\text{macro}}$) | Style Compliance ($\text{SR}_{\text{style}}$) | Latency (s) |
|---|---|---|---|---|---|
gemini-3.1-flash-lite |
Vector + Batch | $0.0\%$ | $100.0\%$ | $0.0\%$ | 5.0 |
| Graph + Chronological | $0.0\%$ | $100.0\%$ | $100.0\%$ | 5.4 | |
GAP (Proposed) |
$100.0\%$ |
$100.0\%$ |
$100.0\%$ |
4.3 |
|
gemini-3.5-flash |
Vector + Batch | $0.0\%$ | $25.0\%$ | $100.0\%$ | 26.9 |
| Graph + Chronological | $0.0\%$ | $50.0\%$ | $100.0\%$ | 48.2 | |
GAP (Proposed) |
$100.0\%$ |
$100.0\%$ |
$100.0\%$ |
50.4 |

Let's break down the metrics:

In our experiments, the Causal Chain Recall ($\text{RR}_{\text{causal}}$) was calculated as a strict logical AND—meaning the causal link was only counted as recovered if *both* the cause and the effect were present in the final summary.

One of the most surprising findings was that the highly capable `gemini-3.5-flash`

scored a mere **25.0%** Macro Coverage under the Vector+Batch pipeline, and **50.0%** under the Graph+Chrono pipeline, whereas the smaller `gemini-3.1-flash-lite`

maintained **100.0%** coverage.

Why? Advanced reasoning models have highly strict internal filters for style rules. When forced to respect the 50-word and list-exclusion limits, `gemini-3.5-flash`

actively chose to prune entire system domains to keep the output grammatically coherent. It chose style over content.

However, when we introduced **GAP**, the semantic anchors acted as attention anchors that restricted the model's pruning boundaries. Under GAP, `gemini-3.5-flash`

maintained **100.0% Macro Coverage** while fully complying with the style rules.

Thanks to the parallel tree-reduction architecture orchestrated by the **Antigravity Python SDK**, the Level 1 leaf summaries are processed concurrently.

`gemini-3.1-flash-lite`

running GAP completed the entire pipeline in just `gemini-3.5-flash`

took For real-time operational systems, where every second of delay leads to financial loss, **GAP paired with a lightweight model (3.1-flash-lite)** represents the ultimate sweet spot: sub-5-second execution, minimal cost, and a lossless 100% causal recall.

Where does this make a difference? We mapped GAP to two critical enterprise operational architectures.

In a microservices environment, a single network glitch can trigger a cascade of thousands of downstream exceptions.

GAP traverses the infrastructure topology graph to group logs by their connection dependencies. In less than 5 seconds, it compresses the entire cascade into a single, cohesive alert message that accurately highlights the root cause, preventing alert fatigue and slashing Mean Time to Detection (MTTD).

Advanced Persistent Threat (APT) attacks execute slowly over weeks, leaving quiet footprints across different servers. Standard security alerts treat these events as isolated noise.

By modeling user accounts, IP connections, and host logs as a graph, GAP links lateral movements. The persistent semantic anchors ensure that critical details—like compromised user accounts and IP addresses—are never discarded, delivering a unified threat summary to security teams.

If you are building production-grade RAG applications, it's time to move beyond flat vector lookups and linear text chaining. When relationships and detail retention matter, hierarchical structures are essential.

By extending the parallel tree-reduction mathematics of the **Pyramid Method** with graph-aware topology and static semantic anchoring, GAP ensures that your LLMs retain 100% of causal context without sacrificing speed.

Stop dumping raw texts into prompts. **Build a Graph-Anchor Pyramid.**

*Credits: Google Cloud credits were provided to support this research.*