{"slug": "show-hn-we-benchmarked-ast-stubbing-trade-offs-on-airflow-and-zulip-ctxfw", "title": "Show HN: We benchmarked AST stubbing trade-offs on Airflow and Zulip (ctxfw)", "summary": "The ctxfw v4.0.0-preview multi-depth topological resolution engine cut context tokens by 74.5% on the zulip/zulip repository, 68.9% on apache/airflow, and 48.9% on PostHog/posthog in benchmarks run 2026-10-02 on Ubuntu 24.04 LTS with Docker 29.1.3. The tool, aimed at autonomous coding agents such as Cursor, Claude Code, and Windsurf, pruned zulip's 328,539 raw graph tokens to 83,913, airflow's 411,010 to 127,882, and posthog's 1,333,525 to 681,273, with P95 latency under 20 ms across all three repositories and a 100% AST parse pass rate. The engine runs local-first with zero external API dependencies or network telemetry calls.", "body_md": "**Target Architecture:** Autonomous Coding Agents (Cursor, Claude Code, Windsurf)\n\n**Execution Environment:** Remote Linux Engine (Ubuntu 24.04 LTS, Docker 29.1.3)\n\n**Engine Version:** `ctxfw` v4.0.0-preview (Branch: `experiment/depth-configurator`)\n\n**Verification Date:** 2026-10-02 06:28:28 UTC\n\nTo validate the multi-depth topological resolution engine (\n\nThe targets span three quintessential software engineering archetypes:\n\n1. **`zulip/zulip`:** Tightly coupled Django monolithic backend with complex ORM inheritance.\n2. **`PostHog/posthog`:** Modern Commercial Open Source (COSS) data platform with dynamic model architectures.\n3. **`apache/airflow`:** Asynchronous distributed orchestration monorepo featuring decoupled Task-SDK definitions (Airflow 3.0 / AIP-44).\n\nTokens measured via standard canonical metric `len(text) // 4`:\n\n| Target Repository | Architectural Archetype | Active Focal | Raw Graph Tokens | Pruned | Context Savings | Latency P95 (D3) | Throughput Memoria | \n|---|---|---|---|---|---|---|---|\n| **`zulip`** | Coupled Django Monolith | `zerver/models/users.py` | **328,539** | **83,913** | **-74.5%** | **11.61 ms** | ~7,227,648 tok/s | \n| **`posthog`** | Modern COSS / Data Engine | `posthog/models/team/team.py` | **1,333,525** | **681,273** | **-48.9%** | **250.29 ms** | ~2,721,891 tok/s | \n| **`airflow`** | Async Distributed Orchestration Monorepo | `airflow-core/src/airflow/models/dag.py` | **411,010** | **127,882** | **-68.9%** | **18.73 ms** | ~6,828,749 tok/s | \n\nA pivotal finding is the compounding token compaction as depth expands from direct dependencies to 3-hop ambient boundaries:\n\n- \n$D_0$ (Focal Target):\n- \n$D_1$ (Direct Interface):`...` stubs, preserving signatures and sanitized raises.\n- \n$D_2$ (Transitive Nominal):\n- \n$D_3$ (Ambient Cartography):`distractor_budget: 150` and$\\le 1,000$ net tokens.\n\n- \n**P95 Latency Ceiling (** Across all three repositories, resolution of the complete multi-depth bundle was accomplished in under$\\le 25 ext{ ms}$ ):**20 ms** , driven by the SQLite WAL batch index and the in-memory L1 write-through cache.\n- \n**Zero-Egress Sovereign Isolation:** Executed with local-first file processing; zero external API dependencies or network telemetry calls.\n- \n**AST Pass Rate:** 100% syntactic validity verified across all generated code stubs (`ast.parse() == True` ).\n\n*Forensic report issued under Heurístico LAB Sovereign Governance Protocol.*\n\n*Immutable cryptographic attestation hash:* `837e90a0d2d97f569f7190da2652d4e578efadf86b71d4a5c3020c6e16bf5bd3`", "url": "https://wpnews.pro/news/show-hn-we-benchmarked-ast-stubbing-trade-offs-on-airflow-and-zulip-ctxfw", "canonical_source": "https://github.com/heuristicolab/ctxfw/blob/main/docs/benchmarks/TRILOGY_EMPIRICAL_BENCHMARK.md", "published_at": "2026-10-06 13:58:36+00:00", "updated_at": "2026-10-06 14:21:17.256336+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-tools"], "entities": ["ctxfw", "zulip/zulip", "PostHog/posthog", "apache/airflow", "Cursor", "Claude Code", "Windsurf", "Ubuntu 24.04 LTS"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-we-benchmarked-ast-stubbing-trade-offs-on-airflow-and-zulip-ctxfw", "markdown": "https://wpnews.pro/news/show-hn-we-benchmarked-ast-stubbing-trade-offs-on-airflow-and-zulip-ctxfw.md", "text": "https://wpnews.pro/news/show-hn-we-benchmarked-ast-stubbing-trade-offs-on-airflow-and-zulip-ctxfw.txt", "jsonld": "https://wpnews.pro/news/show-hn-we-benchmarked-ast-stubbing-trade-offs-on-airflow-and-zulip-ctxfw.jsonld"}}