{"slug": "contextops-an-eslint-like-static-analyzer-for-llm-context", "title": "ContextOps, an ESLint-like static analyzer for LLM context", "summary": "ContextOps, a new open-source static analysis tool for LLM context, has been released. It acts as a deterministic linter that evaluates structural quality—such as redundancy, token waste, and source concentration—before inference, producing a Context Health Score without requiring embeddings or external API calls. The tool aims to make LLM context quality observable and testable, similar to how ESLint enforces code quality in software engineering.", "body_md": "Static analysis for LLM context.\n\nContextOps is a **deterministic, model-independent context linter** for LLM applications.\n\nIt analyzes the context sent to an LLM **before inference** and detects structural problems such as redundancy, token waste, context imbalance, and source concentration. It produces a reproducible **Context Health Score (CHS)** together with actionable diagnostics — without embeddings, model calls, or external services.\n\nThink of ContextOps as **ESLint for LLM context**.\n\nModern software engineering has deterministic quality gates.\n\n- Compilers catch syntax errors.\n- Linters catch code smells.\n- Formatters enforce consistency.\n- Static analyzers detect architectural problems.\n\nLLM applications rarely have an equivalent layer for the context they send into models.\n\nInstead, prompts quietly grow over time:\n\n- duplicated retrieval chunks\n- bloated system prompts\n- runaway conversation history\n- excessive tool output\n- hidden token waste\n\nThese issues increase latency and cost, make model behavior less predictable, and often go unnoticed until production.\n\nContextOps makes context quality **observable, measurable, and testable before inference.**\n\nContextOps evaluates four structural dimensions.\n\n| Dimension | Max Penalty | What It Measures |\n|---|---|---|\nRedundancy |\n30 pts | Lexical duplication across context items |\nDensity |\n30 pts | Token waste from formatting and structural bloat |\nStructure |\n20 pts | Distribution imbalance between context components |\nConcentration |\n20 pts | Over-reliance on a single document or source |\n\nThe result is a deterministic **0–100 Context Health Score** together with detailed findings and suggested fixes.\n\nContextOps intentionally does **not** evaluate:\n\n- prompt engineering quality\n- reasoning ability or hallucinations\n- factual correctness\n- retrieval relevance\n- LLM outputs\n\nIt focuses exclusively on the **structural quality of the context before inference.**\n\n```\ncontextops inspect context.json\nContext Health Score: 81 / 100\n\n✓ Low structural complexity\n✓ Good source diversity\n\nWarnings\n\n• 214 duplicated tokens detected\n• Retrieval occupies 78% of context\n• Two retrieval chunks are near duplicates\n\nEstimated token savings: 12%\n```\n\nUse `--roast`\n\nmode for more candid diagnostics:\n\n```\ncontextops inspect context.json --roast\npip install contextops\n```\n\nRun the interactive demo:\n\n```\ncontextops demo\n```\n\n**Dependencies:** Only `tiktoken`\n\nand `click`\n\n. No GPU, no network, no external API keys required.\n\nSave your LLM payload before sending it to the model.\n\n``` python\nimport json\n\nmessages = [\n    {\"role\": \"system\", \"content\": system_prompt},\n    {\"role\": \"user\", \"content\": user_query},\n]\n\n# Add two lines before your API call\nwith open(\"context.json\", \"w\") as f:\n    json.dump(messages, f, indent=2)\n\nresponse = openai.chat.completions.create(model=\"gpt-4o\", messages=messages)\n```\n\nOr use the structured dict format for richer analysis:\n\n```\n{\n  \"system\": \"You are a helpful customer support bot.\",\n  \"messages\": [\n    {\"role\": \"user\", \"content\": \"How long will my refund take?\"}\n  ],\n  \"chunks\": [\n    {\"content\": \"Refunds take 3-5 business days.\", \"source\": \"docs/refunds.md\"}\n  ],\n  \"memory\": [\"The user asked about a refund yesterday.\"],\n  \"tools\": [{\"name\": \"search_api\", \"output\": \"Tool response text here\"}]\n}\ncontextops inspect context.json\ncontextops check context.json --min-score 75\n```\n\nAnalyse a context file and display a rich report.\n\n```\ncontextops inspect context.json\ncontextops inspect context.json --roast\ncontextops inspect context.json --profile rag\ncontextops inspect context.json --explain\ncontextops inspect context.json --json-output\n```\n\n| Flag | Default | Description |\n|---|---|---|\n`--json-output` |\noff | Output raw JSON instead of terminal format |\n`--model <name>` |\n`gpt-4o` |\nModel for token encoding (e.g. `gpt-4` , `claude-3` ) |\n`--profile <name>` |\n`general` |\nArchetype: `rag` , `agent` , `chatbot` , `toolchain` |\n`--config <path>` |\nnone | Path to a JSON config file with custom thresholds |\n`--retrieval-max-ratio <f>` |\n0.70 | Override max allowed ratio for retrieval chunks |\n`--system-max-ratio <f>` |\n0.50 | Override max allowed ratio for system prompt |\n`--memory-max-ratio <f>` |\n0.50 | Override max allowed ratio for memory entries |\n`--tool-max-ratio <f>` |\n0.60 | Override max allowed ratio for tool outputs |\n`--explain` |\noff | Show detailed \"Top Score Drivers\" for each penalty |\n`--roast` |\noff | Enable brutally honest score-band commentary |\n\nCI gate. Exits with code `0`\n\n(pass) or `1`\n\n(fail).\n\n```\ncontextops check context.json --min-score 75\n```\n\n| Exit Code | Meaning |\n|---|---|\n`0` |\nPASS — score meets or exceeds threshold |\n`1` |\nFAIL — score is below threshold or analysis error |\n`2` |\nERROR — invalid JSON or unreadable input |\n\n**GitHub Actions example:**\n\n```\n- name: Check context quality\n  run: contextops check context.json --min-score 75 --profile rag\n```\n\nCompare two context snapshots to detect regressions.\n\n```\ncontextops diff before.json after.json\n```\n\nShows score delta, which penalties changed, and whether the change is an improvement or regression. Useful for A/B testing retrieval strategies or prompt refactors.\n\nRun a deterministic stability report to verify the scoring engine is working correctly.\n\n```\ncontextops stability\ncontextops stability context.json\n```\n\nLocal-only, opt-in telemetry for tracking context quality trends over time.\n\n```\ncontextops telemetry status          # Check if telemetry is active\ncontextops telemetry log --limit 25  # Show recent events\ncontextops telemetry trends --days 7 # Show 7-day quality trends\n```\n\nGenerate a GitHub shields.io markdown badge.\n\n```\ncontextops badge             # Uses your last telemetry score\ncontextops badge --score 87  # Use a specific score\n```\n\nOutput:\n\n```\n[![ContextOps](https://img.shields.io/badge/ContextOps-87-green)](https://github.com/Abhijeet777/contextops)\npython\nfrom contextops.api.inspect import inspect_context\n\nresult = inspect_context(\n    payload,          # dict, list, or plain string\n    model=\"gpt-4o\",   # model for token counting\n    archetype=\"rag\",  # archetype profile (optional)\n)\n\nprint(result.score)               # int 0–100\nprint(result.token_breakdown.wasted_tokens)\nfor rec in result.recommendations:\n    print(f\"  -> {rec.fix}\")\n```\n\n**Full result fields:**\n\n```\nresult.score                  # int 0–100\nresult.score_breakdown        # redundancy, density, structure, concentration penalties\nresult.token_breakdown        # total_tokens, by_type, wasted_tokens, estimated_cost_usd\nresult.redundancy_findings    # list[RedundancyFinding]\nresult.structure_findings     # list[StructureFinding]\nresult.recommendations        # list[Recommendation]\nresult.density_signal         # DensitySignal\nresult.archetype_resolved     # e.g. \"rag\"\nresult.roast                  # str | None (if roast_enabled)\nresult.metadata               # item_count, model, version\npython\nfrom contextops.api.diff import diff_contexts\n\nresult = diff_contexts(payload_a, payload_b)\npython\nfrom contextops.core.config import ContextOpsConfig\n\nconfig = ContextOpsConfig.default(profile=\"rag\")\n\nconfig = ContextOpsConfig.from_dict({\n    \"retrieval_max_ratio\": 0.90,\n    \"system_max_ratio\": 0.30,\n    \"roast_enabled\": True\n})\npip install contextops langchain-core\npython\nfrom contextops import ContextOps\n\n# Log the score (default — non-blocking)\nchain = chain.with_config({\n    \"callbacks\": [ContextOps.auto()]\n})\n\n# Block execution if context quality is too low\nchain = chain.with_config({\n    \"callbacks\": [\n        ContextOps.auto(\n            mode=\"block\",\n            min_score=75,\n            profile=\"rag\"\n        )\n    ]\n})\n\nresult = chain.invoke({\"question\": \"What is the refund policy?\"})\n```\n\n| Mode | Behaviour |\n|---|---|\n`\"log\"` |\nPrints score report to stdout (default) |\n`\"warn\"` |\nEmits Python warning if score < min_score |\n`\"block\"` |\nRaises `ContextOpsScoreError` if score < min_score — blocks LLM call |\n\nArchetypes adjust structural thresholds for your specific use case. The global 0–100 score is never affected — only which warnings fire.\n\n| Profile | When to Use | Retrieval | System | Memory | Tool |\n|---|---|---|---|---|---|\n`general` |\nDefault — mixed use cases | 70% | 50% | 50% | 60% |\n`rag` |\nPure document retrieval | 95% |\n40% | 20% | 30% |\n`agent` |\nAutonomous agents with tool loops | 50% | 40% | 40% | 90% |\n`chatbot` |\nConversational apps with large history | 40% | 50% | 85% |\n30% |\n`toolchain` |\nMulti-tool pipelines | 50% | 40% | 30% | 95% |\n\n**Resolution order** (highest priority wins):\n\n`--profile`\n\nCLI flag`archetype=`\n\nPython API argument`\"archetype\"`\n\nkey inside the JSON payload`config.context_profile`\n\n- Default:\n`\"general\"`\n\nContextOps accepts three input formats.\n\n**Structured dict** (recommended — gives the richest analysis):\n\n```\n{\n  \"system\": \"...\",\n  \"messages\": [...],\n  \"chunks\": [...],\n  \"memory\": [...],\n  \"tools\": [...]\n}\n```\n\n**OpenAI message list** (paste your existing payload directly):\n\n```\n[\n  {\"role\": \"system\", \"content\": \"You are a helpful bot.\"},\n  {\"role\": \"user\", \"content\": \"Hello\"}\n]\n```\n\n**Plain string** (treated as a system prompt).\n\n```\nScore = max(0, min(100, round(100 − total_penalty)))\n```\n\nWhere `total_penalty = redundancy + density + structure + concentration`\n\n.\n\n| Penalty | Max | What It Detects |\n|---|---|---|\nRedundancy |\n30 | Lexical duplication via Jaccard + N-gram overlap |\nDensity |\n30 | Format overhead, whitespace waste, entropy compression |\nStructure |\n20 | Retrieval dominance, system bloat, memory explosion |\nConcentration |\n20 | Source dominance + entropy imbalance across chunks |\n\nLexical only.ContextOps uses N-gram overlap, Jaccard similarity, and exact string matching — not semantic similarity. This is intentional: it is a structural analyser, not a semantic one.\n\nIf LeetCode is **DSA for algorithms**, ContextBench is **DSA for context**.\n\nJust as competitive programming teaches you that the algorithm matters — not just the answer — ContextBench teaches you that **context architecture matters**, not just whether the LLM eventually gets it right.\n\nA brute-force O(N²) solution that produces the correct answer still gets penalized for time complexity. A bloated, redundant context that still produces a good LLM response still gets penalized for structural waste. The leaderboard notices both.\n\n| LeetCode / DSA | ContextBench Leaderboard |\n|---|---|\n| Problem set | 1,500 pre-built context windows across 5 failure categories |\n| Judge | ContextOps engine — deterministic scorer, same output every time |\n| Your solution | `optimize_context(ctx)` — takes a broken context, returns a better one |\n| Score metric | Quality (50%) + Compression (35%) + Latency (15%) |\n| Leaderboard | Ranked by `final_score` across all benchmark samples |\n| Adversarial track | ContextSecBench — 9,500 adversarial attack payloads |\n\nContextBench contains **1,500 samples** across 5 categories — think of these as your difficulty tiers:\n\n| Category | Samples | What It Tests |\n|---|---|---|\n| Optimal Architectures | 300 | Healthy pipelines — prove you don't produce false positives |\n| Structural Failures | 300 | System prompt bloat, retrieval flooding, memory explosion |\n| Redundancy Failures | 300 | Near-duplicate clusters, boilerplate explosion, paraphrase loops |\n| Agent Architecture Failures | 300 | Multi-agent context explosion, recursive planning loops, tool chain bloat |\n| Temporal Context Drift | 300 | Stale memory injection, retrieval drift, invalidated historical state |\n\nEvery sample has a ground truth — just like a LeetCode problem has expected output:\n\n```\n{\n  \"ground_truth\": {\n    \"failure_modes\": [\"system_prompt_bloat\"],\n    \"expected_properties\": {\n      \"contains_redundancy\": false,\n      \"contains_density_bloat\": true,\n      \"contains_structure_imbalance\": true\n    }\n  }\n}\n```\n\nFlag redundancy when `contains_redundancy: false`\n\n? **False positive. Score drops.** Miss the density bloat? **False negative. Score drops.**\n\nInstead of writing a sorting algorithm, you write a context optimizer:\n\n``` php\ndef optimize_context(ctx: dict) -> dict:\n    # Your logic: prune, compress, deduplicate, rebalance\n    return better_ctx\n```\n\nThe harness runs your function across all 1,500 samples and scores:\n\n```\nfinal_score = (\n    0.50 * quality_score     # ContextOps CHS must stay ≥ 78\n  + 0.35 * compression       # Token reduction reward\n  + 0.15 * latency_mult      # Speed penalty if you're slow\n) * 100\n```\n\nThe **Quality Floor Gate** (score ≥ 78) is non-negotiable — optimizers that destroy context quality to hit compression targets are disqualified.\n\nContextSecBench is the adversarial extension — the CTF track on top of LeetCode.\n\n**9,500 attack payloads** covering:\n\n**Prompt injection hiding**— malicious instructions buried deep inside retrieved context** Truncation smuggling**— payloads designed to survive chunking with harmful content intact** Semantic Denial of Service (SDoS)**— padding the context window to exhaust the attention budget** Context poisoning**— subtle corruption of retrieval content to bias model behavior** Format corruption**— structural attacks that break parser assumptions\n\nYour optimizer must handle malicious context the same way a secure sorting algorithm must handle adversarial inputs.\n\nPoor context architecture causes problems long before the model itself becomes the bottleneck:\n\n- Higher token costs and inference latency\n- Lost-in-the-middle failures on long contexts\n- Degraded reasoning quality from attention bandwidth collapse\n- Context window exhaustion in long-running agent loops\n- Hidden operational waste that compounds silently at scale\n\nBetter models don't solve poor context architecture. ContextBench measures the layer that does.\n\nThe same input always produces the same score, breakdown, findings, and recommendations — on any machine, at any time. No randomness.\n\nNo embeddings, GPUs, inference APIs, or external AI services. The entire engine is pure Python math.\n\nGuaranteed `<2s`\n\nfor payloads ≤ 5,000 tokens. `<10s`\n\nfor 50,000-token payloads. Works offline. Exit-code driven.\n\nMeasures context architecture — not semantic quality, not model correctness, not factual truth.\n\n| Payload Size | Max Execution Time |\n|---|---|\n| ≤ 5,000 tokens | < 2 seconds |\n| ≤ 20,000 tokens | < 5 seconds |\n| ≤ 50,000 tokens | < 10 seconds |\n\nContextOps aims to make LLM context as **observable, measurable, and testable as source code.**\n\nJust as modern software uses compilers, linters, and static analysis before deployment, LLM systems should validate the structural quality of their context before inference.\n\n```\nRetrieve\n      │\n      ▼\n Build Context\n      │\n      ▼\n  ContextOps   ←── structural gate\n      │\n      ▼\n     LLM\n```\n\n| Document | Contents |\n|---|---|\n|\n\n[STABILITY.md](/Abhijeet777ui/contextops/blob/main/STABILITY.md)[USER_GUIDE.md](/Abhijeet777ui/contextops/blob/main/USER_GUIDE.md)[HOW_TO_GET_JSON.md](/Abhijeet777ui/contextops/blob/main/HOW_TO_GET_JSON.md)[CONTRIBUTING.md](/Abhijeet777ui/contextops/blob/main/CONTRIBUTING.md)[CHANGELOG.md](/Abhijeet777ui/contextops/blob/main/CHANGELOG.md)Contributions are welcome — bug fixes, new ContextBench samples, documentation improvements, and new analyzer signals.\n\nRead ** CONTRIBUTING.md** for:\n\n- Development setup and project structure\n- How to run the test suite (including the chaos and signal contract tests)\n- Rules for adding new signals or changing the scoring engine\n- The PR checklist and commit style guide\n- What the stability contract forbids\n\nFor anything non-trivial, open an issue first so we can align before you write code.\n\n**Repository:** [github.com/Abhijeet777/contextops](https://github.com/Abhijeet777/contextops)\n\nReleased under the [Sustainable Use License](/Abhijeet777ui/contextops/blob/main/LICENSE).", "url": "https://wpnews.pro/news/contextops-an-eslint-like-static-analyzer-for-llm-context", "canonical_source": "https://github.com/Abhijeet777ui/contextops", "published_at": "2026-07-11 22:36:06+00:00", "updated_at": "2026-07-11 23:05:27.338386+00:00", "lang": "en", "topics": ["developer-tools", "large-language-models", "ai-tools"], "entities": ["ContextOps", "OpenAI", "GPT-4o"], "alternates": {"html": "https://wpnews.pro/news/contextops-an-eslint-like-static-analyzer-for-llm-context", "markdown": "https://wpnews.pro/news/contextops-an-eslint-like-static-analyzer-for-llm-context.md", "text": "https://wpnews.pro/news/contextops-an-eslint-like-static-analyzer-for-llm-context.txt", "jsonld": "https://wpnews.pro/news/contextops-an-eslint-like-static-analyzer-for-llm-context.jsonld"}}