{"slug": "contextfusion-the-context-brain-your-llm-apps-are-missing", "title": "ContextFusion: The Context Brain Your LLM Apps Are Missing", "summary": "ContextFusion, an open-source middleware tool from developer rotsl, claims to reduce LLM token usage by 60–99% while maintaining answer quality, using a multi-objective knapsack optimizer and delta fusion for agent conversations. Benchmarks were run with Claude Sonnet 4.6 on production-like workloads, and the tool supports providers like OpenAI and Anthropic, with installation via npm or pip.", "body_md": "Normal users: Install context-portfolio-optimizer, run cpo compile ./your-docs --budget 4000, and stop overpaying for tokens.\n\nDevelopers: Middleware pipeline that ingests heterogeneous sources → normalizes → precomputes → optimizes via multi-objective knapsack → compiles provider-specific payloads with delta fusion for agents.\n\nBoth groups get 60–99% token reduction with identical answer quality.\n\nYou’re building with LLMs. Maybe it’s a chatbot over your company docs. Maybe it’s a coding assistant. Maybe it’s an agent that needs to remember context across 20 turns.\n\nYou keep hitting the same frustrations:\n\n*You’ve tried RAG. You’ve tried chunking. But you’re still blindly stuffing retrieved chunks into prompts without knowing which ones actually matter.*\n\nThink of it like a smart travel packer for your LLM trips.\n\nYou have a weight limit (token budget). You have dozens of items (documents, code, images). Some items are essential. Some are nice-to-have. Some are duplicates. Some are risky (outdated, untrusted).\n\nContextFusion:\n\n*Benchmarks run with Claude Sonnet 4.6 on production-like workloads. Full methodology at **github.com/rotsl/context-fusion/benchmarks*\n\n```\n# One-time setupnpm install -g @rotsl/contextfusionnpx @rotsl/contextfusion setup# Create API keys filenpx @rotsl/contextfusion env# Edit .env with your OPENAI_API_KEY or ANTHROPIC_API_KEY# Run optimizationnpx @rotsl/contextfusion run ./my-documents \\  --query \"Summarize key findings\" \\  --provider anthropic \\  --model claude-sonnet-4-6 \\  --budget 4000# Launch Web UInpx @rotsl/contextfusion ui --port 8080\npip install context-portfolio-optimizer# Set up environmentcat > .env << 'EOF'ANTHROPIC_API_KEY=your_key_hereOPENAI_API_KEY=your_key_hereEOF# Run CLIcpo run ./my-documents --budget 4000 --query \"What are the main points?\"# Or compile for specific task typecpo compile ./my-codebase \\  --task \"Explain this function\" \\  --provider openai \\  --model gpt-5-mini \\  --mode code \\  --budget 3000\ndocker build -t context-fusion:latest .docker run --rm -it -v \"$(pwd)\":/app context-fusion:latest run ./data --budget 3000\n```\n\nRun cpo ui --port 8080 and open your browser. You'll see:\n\n*This transparency is rare. Most RAG tools are black boxes. ContextFusion shows its work.*\n\n✅ Multi-provider setups — Same pipeline, different output formats\n\n✅ Cost-sensitive production — 60–99% token reduction\n\n✅ Agent conversations — Delta fusion prevents token churn\n\n✅ Complex ingestion — PDFs, images, code, spreadsheets unified\n\n✅ Latency requirements — Precomputation + caching\n\n❌ Simple single-turn Q&A with tiny documents\n\n❌ You’re already heavily invested in a specific RAG framework and happy with costs\n\n❌ You need real-time streaming with sub-100ms latency (ContextFusion adds 50–200ms optimization overhead)\n\n```\n┌─────────────────────────────────────────────────────────────────┐│                        INGESTION LAYER                          ││  PDF │ DOCX │ CSV │ JSON │ Images (OCR) │ Code │ Markdown       │└─────────────────────────────────────────────────────────────────┘                              ↓┌─────────────────────────────────────────────────────────────────┐│                     NORMALIZATION LAYER                         ││  Convert all sources to uniform ContextBlock objects            ││  - source_type, content_hash, created_at, metadata              │└─────────────────────────────────────────────────────────────────┘                              ↓┌─────────────────────────────────────────────────────────────────┐│                   REPRESENTATION LAYER                          ││  Precompute compact variants per block:                         ││  - universal_summary (general purpose)                          ││  - qa_extractive (question-answering focused)                   ││  - code_signature (functions, classes, dependencies)            ││  - agent_condensed (working memory format)                      │└─────────────────────────────────────────────────────────────────┘                              ↓┌─────────────────────────────────────────────────────────────────┐│                    PRECOMPUTE PIPELINE                          ││  Store: fingerprints, summaries, token stats,                     ││  retrieval features, compact variants in .cpo_cache/            │└─────────────────────────────────────────────────────────────────┘                              ↓┌─────────────────────────────────────────────────────────────────┐│                    RETRIEVAL LAYER                              ││  Query classification → Lexical retrieval (top-100)             ││  → Fast rerank (top-20/25) → Candidate set                    │└─────────────────────────────────────────────────────────────────┘                              ↓┌─────────────────────────────────────────────────────────────────┐│              MULTI-OBJECTIVE PLANNER (Core)                     ││                                                                 ││  maximize Σ( w_u·utility - w_r·risk - w_t·token_cost           ││           - w_l·latency + w_c·cacheability + w_d·diversity )  ││                                                                 ││  subject to: Σ(token_i) ≤ budget                               ││                                                                 ││  Selects optimal representation variant per block               │└─────────────────────────────────────────────────────────────────┘                              ↓┌─────────────────────────────────────────────────────────────────┐│                   COMPRESSION LAYER                             ││  - JSON minification                                            ││  - Citation compaction (Source URI → [id])                      ││  - Schema field pruning                                         ││  Levels: none │ light │ medium │ aggressive                     │└─────────────────────────────────────────────────────────────────┘                              ↓┌─────────────────────────────────────────────────────────────────┐│                    DELTA FUSION (Agent Mode)                    ││  Compute ContextDelta:                                          ││  - added_blocks: new since last turn                            ││  - updated_blocks: changed content                              ││  - removed_blocks: no longer relevant                           ││  - unchanged_block_ids: reuse from cache                        │└─────────────────────────────────────────────────────────────────┘                              ↓┌─────────────────────────────────────────────────────────────────┐│              PROVIDER ADAPTER LAYER                             ││  Compile provider-specific payloads:                            ││  - openai: chat.completions format                              ││  - anthropic: messages with XML citations                       ││  - ollama: local API structure                                  ││  - openai_compatible: generic wrapper                           │└─────────────────────────────────────────────────────────────────┘                              ↓┌─────────────────────────────────────────────────────────────────┐│                 CACHE-AWARE ASSEMBLY                            ││  Segment into:                                                  ││  - stable: system instructions, citation maps, cacheable blocks ││  - dynamic: volatile content, real-time data                    │└─────────────────────────────────────────────────────────────────┘\n```\n\nMost RAG tools use semantic similarity: embed query, embed chunks, return top-k. This fails when:\n\n*ContextFusion’s planner treats this as a constrained optimization problem:*\n\n```\n# Pseudocode of the core algorithmdef select_context_blocks(candidates, budget, weights):    \"\"\"    candidates: List[ContextBlock with multiple representation variants]    budget: int (token limit)    weights: dict[str, float] (utility, risk, latency, cacheability, diversity)    \"\"\"        # Generate all (block, variant) pairs with scores    items = []    for block in candidates:        for variant in block.representations:            score = (                weights['utility'] * variant.utility_score                - weights['risk'] * block.risk_score                - weights['token_cost'] * variant.token_count                - weights['latency'] * variant.latency_estimate                + weights['cacheability'] * block.cache_score                + weights['diversity'] * diversity_bonus(block, selected)            )            items.append((block.id, variant, score, variant.token_count))        # Solve 0/1 knapsack for maximum score within budget    selected = knapsack_01(items, budget)    return selected\n```\n\n*This is NP-hard, but with proper indexing and heuristics, it runs in <100ms for typical workloads.*\n\n``` python\nfrom context_portfolio_optimizer import PipelineRunner, Configfrom context_portfolio_optimizer.providers import AnthropicAdapter# Custom configurationconfig = Config.from_yaml(\"\"\"budget:  instructions: 1000  retrieval: 3000  memory: 2000  examples: 1500  tool_trace: 1000  output_reserve: 1000scoring:  utility_weights:    retrieval: 0.25    trust: 0.20    freshness: 0.15    structure: 0.15    diversity: 0.15    token_cost: -0.10provider:  name: anthropic  model: claude-sonnet-4-6\"\"\")# Initialize pipelinerunner = PipelineRunner(config=config)# Run full pipelineresult = runner.run(    sources=[\"./docs/architecture.pdf\", \"./src/api.py\", \"./data/metrics.csv\"],    query=\"How does the authentication flow work?\",    task_mode=\"qa\",  # chat | qa | code | agent    budget=4000,    use_precomputed=True,    compute_delta=False  # Set True for agent loops)# Inspect resultsprint(f\"Selected {result['stats']['blocks_selected']} blocks\")print(f\"Total tokens: {result['stats']['total_tokens']}\")print(f\"Context preview:\\n{result['context'][:500]}...\")# Direct provider compilationadapter = AnthropicAdapter(config.provider)payload = adapter.compile_packet(    context_blocks=result['selected_blocks'],    task=\"Answer with citations\",    model=\"claude-sonnet-4-6\")# payload is ready for anthropic.messages.create(**payload)\n```\n\nStandard agent implementations re-send the entire conversation history + retrieved context on every turn. With 10 turns × 4,000 tokens = 40,000 tokens wasted.\n\n*ContextFusion’s delta tracking:*\n\n```\n# Turn 1: Full contextturn1_result = runner.run(sources, query=\"Step 1...\", task_mode=\"agent\")turn1_packet = turn1_result['context_packet']# Turn 2: Only send what changedturn2_result = runner.run(    sources,     query=\"Step 2...\",    task_mode=\"agent\",    previous_packet=turn1_packet,  # Enable delta computation    compute_delta=True)# turn2_result['context_delta'] contains:# {#   'added_blocks': [new_retrieved_content],#   'updated_blocks': [changed_blocks],#   'removed_blocks': [no_longer_relevant],#   'unchanged_block_ids': [ids_to_reuse_from_cache],#   'full_context_hash': 'abc123...'  # For cache validation# }\n```\n\nThe provider adapter assembles:\n\nFor production workloads, precompute expensive operations:\n\n```\n# One-time setup (can run offline, on CI, or scheduled)cpo precompute ./corpus \\  --store-dir .cpo_cache/precompute \\  --semantic-dedup \\  --generate-all-representations# Runtime query uses precomputed artifactscpo compile ./corpus \\  --precomputed-only \\  --query \"Quick question\" \\  --budget 2000\n```\n\nPrecomputed artifacts:\n\n*Expose ContextFusion as an MCP (Model Context Protocol) server:*\n\n```\ncpo serve-mcp --host localhost --port 8765\n```\n\nMCP clients can now call:\n\n*LangChain:*\n\n``` python\nfrom context_portfolio_optimizer.integrations import ContextFusionRetrieverretriever = ContextFusionRetriever(    sources=[\"./docs\"],    budget=3000,    task_mode=\"qa\")# Use in any LangChain chainfrom langchain.chains import RetrievalQAqa = RetrievalQA.from_chain_type(    llm=chat_model,    chain_type=\"stuff\",    retriever=retriever)\n```\n\n*LlamaIndex:*\n\n``` python\nfrom context_portfolio_optimizer.integrations import ContextFusionNodeParserparser = ContextFusionNodeParser(    budget_per_query=4000,    precompute_dir=\".cpo_cache\")# Use with LlamaIndex index constructionfrom llama_index.core import VectorStoreIndexindex = VectorStoreIndex.from_documents(    documents,    node_parser=parser)\ngit clone https://github.com/rotsl/context-fusion.gitcd context-fusionmake bootstrap  # Install dev dependencies# Development workflowmake test        # Run test suite (49 tests)make lint        # Ruff + mypymake type-check  # Strict type checkingmake format      # Auto-format code# Local serversmake ui          # Web UI on :8080make serve-mcp   # MCP server on :8765# Benchmarkingmake benchmark   # Run full benchmark suite\n```\n\n*Custom representation:*\n\n``` python\nfrom context_portfolio_optimizer.representations import Representation, register_representation@register_representation(\"my_custom\")class MyCustomRepresentation(Representation):    def generate(self, block: ContextBlock) -> str:        # Your custom summarization logic        return custom_summarize(block.content)        def estimate_tokens(self, text: str) -> int:        return len(text.split()) * 1.3  # Rough heuristic\n```\n\n*Custom provider adapter:*\n\n``` python\nfrom context_portfolio_optimizer.providers import BaseProviderAdapter, register_adapter@register_adapter(\"my_provider\")class MyProviderAdapter(BaseProviderAdapter):    def compile_packet(self, context_blocks, task, model, **kwargs):        # Format for your custom LLM API        return {            \"model\": model,            \"messages\": [                {\"role\": \"system\", \"content\": self.format_system()},                {\"role\": \"user\", \"content\": self.format_context(context_blocks, task)}            ]        }\n```\n\n*Q: How is this different from LangChain’s **ContextualCompressionRetriever?*\n\n*LangChain’s version compresses after retrieval using an LLM call. ContextFusion optimizes which content to retrieve and which representation to use, without requiring an LLM for compression. It’s also provider-agnostic and handles delta fusion for agents.*\n\n*Q: Does this replace my vector database?*\n\n*No. ContextFusion sits after retrieval. Use Pinecone, Weaviate, pgvector, or FAISS for initial retrieval — then pass candidates through ContextFusion for optimization.*\n\n*Q: What about streaming responses?*\n\n*ContextFusion optimizes the input context. Streaming the LLM’s output is unaffected. The optimization adds 50–200ms overhead, which is usually offset by reduced LLM latency from shorter prompts.*\n\n*Q: Can I use this with local models?*\n\n*Yes. The Ollama adapter works with any OpenAI-compatible local server. Budget planning and compression are even more valuable with slower local hardware.*\n\n*Q: How do I debug suboptimal context selection?*\n\n*Run **cpo ui and inspect the \"Selected Blocks\" panel. Each block shows its utility score, risk score, token count, and why it was included/excluded. Run **cpo ablate ./data to see which blocks contribute most to answer quality.*\n\nContextFusion isn’t just another RAG tool. It’s a bet that context optimization — treating token budgets as scarce resources to be allocated intelligently — will become as essential as retrieval itself.\n\n**For normal users:** Install it, run it, pay less.\n\n**For developers:** Extend it, integrate it, build smarter systems.\n\nFuse less context. Keep more signal. Ship faster answers.\n\n*⭐️ Star the repo, ⚠️file issues, ㊣ submit PRs. ContextFusion is Apache-2.0 and built for production.*\n\n[ContextFusion: The Context Brain Your LLM Apps Are Missing](https://pub.towardsai.net/contextfusion-the-context-brain-your-llm-apps-are-missing-c2fd9e632b99) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/contextfusion-the-context-brain-your-llm-apps-are-missing", "canonical_source": "https://pub.towardsai.net/contextfusion-the-context-brain-your-llm-apps-are-missing-c2fd9e632b99?source=rss----98111c9905da---4", "published_at": "2026-09-01 05:07:30+00:00", "updated_at": "2026-09-01 05:23:03.868079+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "large-language-models", "ai-infrastructure"], "entities": ["ContextFusion", "rotsl", "Claude Sonnet 4.6", "OpenAI", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/contextfusion-the-context-brain-your-llm-apps-are-missing", "markdown": "https://wpnews.pro/news/contextfusion-the-context-brain-your-llm-apps-are-missing.md", "text": "https://wpnews.pro/news/contextfusion-the-context-brain-your-llm-apps-are-missing.txt", "jsonld": "https://wpnews.pro/news/contextfusion-the-context-brain-your-llm-apps-are-missing.jsonld"}}