# Show HN: MemStitch – Zero-copy context bridging for vLLM (25x TTFT speedup)

> Source: <https://github.com/DaqulaLin/MemStitch>
> Published: 2026-07-14 01:04:02+00:00

**Zero-Copy Context Bridging Gateway for Multi-Agent GPU Inference.**

In multi-agent collaborative workflows, separate agents often process the same long text context sequentially. For example:

**Agent A (Legal Auditor)**: Reads a 200-page contract and runs compliance analyses (populating the GPU KV Cache).** Agent B (Financial Compliance)**: Reads the same 200-page contract and audits financial liabilities.

Under standard inference engines, **Agent B is forced to repeat the expensive prefill phase**, duplicate GPU activations, and suffer from high Time-to-First-Token (TTFT) latency.

**Context-Stitcher** solves this by bridging caches at the memory level:

**Context Topological Hashing**: Segmenting prompts into physical block-sizes and mapping them to cryptographic fingerprints (Merkle-chains).** Zero-Copy Block Stitching**: Bypassing prefill for matched prefixes by mapping the logical attention table of Agent B directly to the physical GPU memory address of Agent A's cache blocks.**Zero-Trust Secure Gate**: Enforcing boundary control lists so unauthorized agent sessions cannot access shared physical blocks.

Below is the benchmark analysis of Context-Stitcher compared to standard vLLM cold-prefills when executing consecutive agents over a shared 200-page document:

```
⚡ TTFT Prefill Latency (Agent B Response Time) — Lower is better
Baseline (vLLM Cold):   ██████████████████████████████  1200 ms
Context-Stitcher:       █  48 ms ( 25.0x Prefill Speedup! 🚀 )

💾 GPU Physical Cache Blocks Allocated (Total VRAM) — Lower is better
Baseline (vLLM Cold):   ██████████████████████████████  53 blocks (No sharing)
Context-Stitcher:       ████████████████  30 blocks ( 43.4% Memory Saved! 📉 )
pip install -r requirements.txt
python run.py
```

Once booted, the gateway routes are active on `http://localhost:8000`

:

**API Proxy Gateway**:`http://localhost:8000/v1/chat/completions`

**Real-time Developer Console**: Open`http://localhost:8000`

in your web browser.

Context-Stitcher includes a responsive developer portal to monitor physical cache block states (idle, private allocations, shared/stitched pages, security alarms) in real time.

Context-Stitcher supports **Python SDK Decorators** and **OpenAI-Compatible REST APIs** for cross-application integrations:

If your agent pipelines are written in Python, you can utilize the `StitcherMesh`

and `@stitch_agent`

decorators to link context memory:

``` python
from context_stitcher import StitcherMesh, stitch_agent

# 1. Initialize the pointer-sharing inference control mesh (ref: vLLM)
mesh = StitcherMesh(backend="vllm", model="meta-llama/Llama-3.1-8B-Instruct")

# Configure secure gate policy: allow Agent B to read Agent A's KV Cache blocks
mesh.sg.add_policy("agent_a", "agent_b")

# 2. Decorate your agent workflows
@stitch_agent(mesh)
def agent_a():
    prompt = "[Long Document Context...]\nAnalyze the intellectual property clauses."
    # First execution: Prefills the cache and registers topological hashes
    res = mesh.generate(prompt=prompt, fingerprint="legal_doc_v1")
    return "legal_doc_v1"

@stitch_agent(mesh)
def agent_b(context_fingerprint):
    prompt = "[Long Document Context...]\nEvaluate the financial compliance risk."
    # Subsequent execution: Automatically stitches blocks, bypassing prefill phase
    res = mesh.generate(prompt=prompt, fingerprint=context_fingerprint)
    print(f"Agent B Response: {res['generated_text']}")
    print(f"Time Saved: {res['prefill_time_saved_ms']}ms")
```

The gateway exposes standard OpenAI endpoints. Point your LLM client base URL to Context-Stitcher to activate sharing.

``` python
from openai import OpenAI

# Direct client to the Context-Stitcher proxy gateway
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")

# Send standard completions request with agent_id metadata inside extra_body
response = client.chat.completions.create(
    model="context-stitcher-sim",
    messages=[
        {"role": "user", "content": "[Long Document Context...]\nEvaluate compliance risk."}
    ],
    extra_body={
        "agent_id": "AgentB",                # Identifies the requesting Agent
        "session_id": "session_legal_audit"  # Identifies the shared session cache
    }
)

print("Generated Output:", response.choices[0].message.content)
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "context-stitcher-sim",
    "messages": [
      {"role": "user", "content": "[Long Document Context...]\nEvaluate compliance risk."}
    ],
    "agent_id": "AgentB",
    "session_id": "session_legal_audit"
  }'
```

You can inspect, add, or revoke access authorization rules dynamically between agents.

```
curl -X GET http://localhost:8000/policies
curl -X POST http://localhost:8000/policy \
  -H "Content-Type: application/json" \
  -d '{"owner_agent": "AgentA", "allowed_reader": "AgentB"}'
curl -X DELETE http://localhost:8000/policy \
  -H "Content-Type: application/json" \
  -d '{"owner_agent": "AgentA", "allowed_reader": "AgentB"}'
```