# ContextFusion: The Context Brain Your LLM Apps Are Missing > Source: > Published: 2026-09-01 05:07:30+00:00 Normal users: Install context-portfolio-optimizer, run cpo compile ./your-docs --budget 4000, and stop overpaying for tokens. Developers: Middleware pipeline that ingests heterogeneous sources → normalizes → precomputes → optimizes via multi-objective knapsack → compiles provider-specific payloads with delta fusion for agents. Both groups get 60–99% token reduction with identical answer quality. You’re building with LLMs. Maybe it’s a chatbot over your company docs. Maybe it’s a coding assistant. Maybe it’s an agent that needs to remember context across 20 turns. You keep hitting the same frustrations: *You’ve tried RAG. You’ve tried chunking. But you’re still blindly stuffing retrieved chunks into prompts without knowing which ones actually matter.* Think of it like a smart travel packer for your LLM trips. You have a weight limit (token budget). You have dozens of items (documents, code, images). Some items are essential. Some are nice-to-have. Some are duplicates. Some are risky (outdated, untrusted). ContextFusion: *Benchmarks run with Claude Sonnet 4.6 on production-like workloads. Full methodology at **github.com/rotsl/context-fusion/benchmarks* ``` # One-time setupnpm install -g @rotsl/contextfusionnpx @rotsl/contextfusion setup# Create API keys filenpx @rotsl/contextfusion env# Edit .env with your OPENAI_API_KEY or ANTHROPIC_API_KEY# Run optimizationnpx @rotsl/contextfusion run ./my-documents \ --query "Summarize key findings" \ --provider anthropic \ --model claude-sonnet-4-6 \ --budget 4000# Launch Web UInpx @rotsl/contextfusion ui --port 8080 pip install context-portfolio-optimizer# Set up environmentcat > .env << 'EOF'ANTHROPIC_API_KEY=your_key_hereOPENAI_API_KEY=your_key_hereEOF# Run CLIcpo run ./my-documents --budget 4000 --query "What are the main points?"# Or compile for specific task typecpo compile ./my-codebase \ --task "Explain this function" \ --provider openai \ --model gpt-5-mini \ --mode code \ --budget 3000 docker build -t context-fusion:latest .docker run --rm -it -v "$(pwd)":/app context-fusion:latest run ./data --budget 3000 ``` Run cpo ui --port 8080 and open your browser. You'll see: *This transparency is rare. Most RAG tools are black boxes. ContextFusion shows its work.* ✅ Multi-provider setups — Same pipeline, different output formats ✅ Cost-sensitive production — 60–99% token reduction ✅ Agent conversations — Delta fusion prevents token churn ✅ Complex ingestion — PDFs, images, code, spreadsheets unified ✅ Latency requirements — Precomputation + caching ❌ Simple single-turn Q&A with tiny documents ❌ You’re already heavily invested in a specific RAG framework and happy with costs ❌ You need real-time streaming with sub-100ms latency (ContextFusion adds 50–200ms optimization overhead) ``` ┌─────────────────────────────────────────────────────────────────┐│ INGESTION LAYER ││ PDF │ DOCX │ CSV │ JSON │ Images (OCR) │ Code │ Markdown │└─────────────────────────────────────────────────────────────────┘ ↓┌─────────────────────────────────────────────────────────────────┐│ NORMALIZATION LAYER ││ Convert all sources to uniform ContextBlock objects ││ - source_type, content_hash, created_at, metadata │└─────────────────────────────────────────────────────────────────┘ ↓┌─────────────────────────────────────────────────────────────────┐│ REPRESENTATION LAYER ││ Precompute compact variants per block: ││ - universal_summary (general purpose) ││ - qa_extractive (question-answering focused) ││ - code_signature (functions, classes, dependencies) ││ - agent_condensed (working memory format) │└─────────────────────────────────────────────────────────────────┘ ↓┌─────────────────────────────────────────────────────────────────┐│ PRECOMPUTE PIPELINE ││ Store: fingerprints, summaries, token stats, ││ retrieval features, compact variants in .cpo_cache/ │└─────────────────────────────────────────────────────────────────┘ ↓┌─────────────────────────────────────────────────────────────────┐│ RETRIEVAL LAYER ││ Query classification → Lexical retrieval (top-100) ││ → Fast rerank (top-20/25) → Candidate set │└─────────────────────────────────────────────────────────────────┘ ↓┌─────────────────────────────────────────────────────────────────┐│ MULTI-OBJECTIVE PLANNER (Core) ││ ││ maximize Σ( w_u·utility - w_r·risk - w_t·token_cost ││ - w_l·latency + w_c·cacheability + w_d·diversity ) ││ ││ subject to: Σ(token_i) ≤ budget ││ ││ Selects optimal representation variant per block │└─────────────────────────────────────────────────────────────────┘ ↓┌─────────────────────────────────────────────────────────────────┐│ COMPRESSION LAYER ││ - JSON minification ││ - Citation compaction (Source URI → [id]) ││ - Schema field pruning ││ Levels: none │ light │ medium │ aggressive │└─────────────────────────────────────────────────────────────────┘ ↓┌─────────────────────────────────────────────────────────────────┐│ DELTA FUSION (Agent Mode) ││ Compute ContextDelta: ││ - added_blocks: new since last turn ││ - updated_blocks: changed content ││ - removed_blocks: no longer relevant ││ - unchanged_block_ids: reuse from cache │└─────────────────────────────────────────────────────────────────┘ ↓┌─────────────────────────────────────────────────────────────────┐│ PROVIDER ADAPTER LAYER ││ Compile provider-specific payloads: ││ - openai: chat.completions format ││ - anthropic: messages with XML citations ││ - ollama: local API structure ││ - openai_compatible: generic wrapper │└─────────────────────────────────────────────────────────────────┘ ↓┌─────────────────────────────────────────────────────────────────┐│ CACHE-AWARE ASSEMBLY ││ Segment into: ││ - stable: system instructions, citation maps, cacheable blocks ││ - dynamic: volatile content, real-time data │└─────────────────────────────────────────────────────────────────┘ ``` Most RAG tools use semantic similarity: embed query, embed chunks, return top-k. This fails when: *ContextFusion’s planner treats this as a constrained optimization problem:* ``` # Pseudocode of the core algorithmdef select_context_blocks(candidates, budget, weights): """ candidates: List[ContextBlock with multiple representation variants] budget: int (token limit) weights: dict[str, float] (utility, risk, latency, cacheability, diversity) """ # Generate all (block, variant) pairs with scores items = [] for block in candidates: for variant in block.representations: score = ( weights['utility'] * variant.utility_score - weights['risk'] * block.risk_score - weights['token_cost'] * variant.token_count - weights['latency'] * variant.latency_estimate + weights['cacheability'] * block.cache_score + weights['diversity'] * diversity_bonus(block, selected) ) items.append((block.id, variant, score, variant.token_count)) # Solve 0/1 knapsack for maximum score within budget selected = knapsack_01(items, budget) return selected ``` *This is NP-hard, but with proper indexing and heuristics, it runs in <100ms for typical workloads.* ``` python from context_portfolio_optimizer import PipelineRunner, Configfrom context_portfolio_optimizer.providers import AnthropicAdapter# Custom configurationconfig = Config.from_yaml("""budget: instructions: 1000 retrieval: 3000 memory: 2000 examples: 1500 tool_trace: 1000 output_reserve: 1000scoring: utility_weights: retrieval: 0.25 trust: 0.20 freshness: 0.15 structure: 0.15 diversity: 0.15 token_cost: -0.10provider: name: anthropic model: claude-sonnet-4-6""")# Initialize pipelinerunner = PipelineRunner(config=config)# Run full pipelineresult = runner.run( sources=["./docs/architecture.pdf", "./src/api.py", "./data/metrics.csv"], query="How does the authentication flow work?", task_mode="qa", # chat | qa | code | agent budget=4000, use_precomputed=True, compute_delta=False # Set True for agent loops)# Inspect resultsprint(f"Selected {result['stats']['blocks_selected']} blocks")print(f"Total tokens: {result['stats']['total_tokens']}")print(f"Context preview:\n{result['context'][:500]}...")# Direct provider compilationadapter = AnthropicAdapter(config.provider)payload = adapter.compile_packet( context_blocks=result['selected_blocks'], task="Answer with citations", model="claude-sonnet-4-6")# payload is ready for anthropic.messages.create(**payload) ``` Standard agent implementations re-send the entire conversation history + retrieved context on every turn. With 10 turns × 4,000 tokens = 40,000 tokens wasted. *ContextFusion’s delta tracking:* ``` # Turn 1: Full contextturn1_result = runner.run(sources, query="Step 1...", task_mode="agent")turn1_packet = turn1_result['context_packet']# Turn 2: Only send what changedturn2_result = runner.run( sources, query="Step 2...", task_mode="agent", previous_packet=turn1_packet, # Enable delta computation compute_delta=True)# turn2_result['context_delta'] contains:# {# 'added_blocks': [new_retrieved_content],# 'updated_blocks': [changed_blocks],# 'removed_blocks': [no_longer_relevant],# 'unchanged_block_ids': [ids_to_reuse_from_cache],# 'full_context_hash': 'abc123...' # For cache validation# } ``` The provider adapter assembles: For production workloads, precompute expensive operations: ``` # One-time setup (can run offline, on CI, or scheduled)cpo precompute ./corpus \ --store-dir .cpo_cache/precompute \ --semantic-dedup \ --generate-all-representations# Runtime query uses precomputed artifactscpo compile ./corpus \ --precomputed-only \ --query "Quick question" \ --budget 2000 ``` Precomputed artifacts: *Expose ContextFusion as an MCP (Model Context Protocol) server:* ``` cpo serve-mcp --host localhost --port 8765 ``` MCP clients can now call: *LangChain:* ``` python from context_portfolio_optimizer.integrations import ContextFusionRetrieverretriever = ContextFusionRetriever( sources=["./docs"], budget=3000, task_mode="qa")# Use in any LangChain chainfrom langchain.chains import RetrievalQAqa = RetrievalQA.from_chain_type( llm=chat_model, chain_type="stuff", retriever=retriever) ``` *LlamaIndex:* ``` python from context_portfolio_optimizer.integrations import ContextFusionNodeParserparser = ContextFusionNodeParser( budget_per_query=4000, precompute_dir=".cpo_cache")# Use with LlamaIndex index constructionfrom llama_index.core import VectorStoreIndexindex = VectorStoreIndex.from_documents( documents, node_parser=parser) git clone https://github.com/rotsl/context-fusion.gitcd context-fusionmake bootstrap # Install dev dependencies# Development workflowmake test # Run test suite (49 tests)make lint # Ruff + mypymake type-check # Strict type checkingmake format # Auto-format code# Local serversmake ui # Web UI on :8080make serve-mcp # MCP server on :8765# Benchmarkingmake benchmark # Run full benchmark suite ``` *Custom representation:* ``` python from context_portfolio_optimizer.representations import Representation, register_representation@register_representation("my_custom")class MyCustomRepresentation(Representation): def generate(self, block: ContextBlock) -> str: # Your custom summarization logic return custom_summarize(block.content) def estimate_tokens(self, text: str) -> int: return len(text.split()) * 1.3 # Rough heuristic ``` *Custom provider adapter:* ``` python from context_portfolio_optimizer.providers import BaseProviderAdapter, register_adapter@register_adapter("my_provider")class MyProviderAdapter(BaseProviderAdapter): def compile_packet(self, context_blocks, task, model, **kwargs): # Format for your custom LLM API return { "model": model, "messages": [ {"role": "system", "content": self.format_system()}, {"role": "user", "content": self.format_context(context_blocks, task)} ] } ``` *Q: How is this different from LangChain’s **ContextualCompressionRetriever?* *LangChain’s version compresses after retrieval using an LLM call. ContextFusion optimizes which content to retrieve and which representation to use, without requiring an LLM for compression. It’s also provider-agnostic and handles delta fusion for agents.* *Q: Does this replace my vector database?* *No. ContextFusion sits after retrieval. Use Pinecone, Weaviate, pgvector, or FAISS for initial retrieval — then pass candidates through ContextFusion for optimization.* *Q: What about streaming responses?* *ContextFusion optimizes the input context. Streaming the LLM’s output is unaffected. The optimization adds 50–200ms overhead, which is usually offset by reduced LLM latency from shorter prompts.* *Q: Can I use this with local models?* *Yes. The Ollama adapter works with any OpenAI-compatible local server. Budget planning and compression are even more valuable with slower local hardware.* *Q: How do I debug suboptimal context selection?* *Run **cpo ui and inspect the "Selected Blocks" panel. Each block shows its utility score, risk score, token count, and why it was included/excluded. Run **cpo ablate ./data to see which blocks contribute most to answer quality.* ContextFusion isn’t just another RAG tool. It’s a bet that context optimization — treating token budgets as scarce resources to be allocated intelligently — will become as essential as retrieval itself. **For normal users:** Install it, run it, pay less. **For developers:** Extend it, integrate it, build smarter systems. Fuse less context. Keep more signal. Ship faster answers. *⭐️ Star the repo, ⚠️file issues, ㊣ submit PRs. ContextFusion is Apache-2.0 and built for production.* [ContextFusion: The Context Brain Your LLM Apps Are Missing](https://pub.towardsai.net/contextfusion-the-context-brain-your-llm-apps-are-missing-c2fd9e632b99) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.