{"slug": "two-llm-calls-beat-one-67-cheaper-and-100-vs-72-extraction-accuracy", "title": "Two LLM calls beat one: 67% cheaper and 100% vs. 72% extraction accuracy", "summary": "A new experimental framework called Semantic Thermodynamics found that a two-stage LLM architecture using a semantic micro-router before a large executor outperforms a single large-model call, cutting extraction costs by 67.54% and improving accuracy to 100% from 72%. The study, which ran 50 preregistered paired observations on OpenAI's GPT-4o models, showed the two-stage pipeline reduced total token consumption by 74.47% and mean end-to-end latency by 60.73% compared to a single GPT-4o call with a large system prompt.", "body_md": "This repository contains the papers, source code, raw telemetry, and reproducibility artifacts for **Semantic Thermodynamics**, an experimental framework for studying how semantic constraint affects inference cost, latency, output entropy, and reliability in Large Language Models.\n\nThe central engineering premise is simple:\n\nPrompt structure is not merely an interface concern. It changes the computational behavior of inference.\n\nThe research program studies this relationship through **Narrative Gravity**, **Narrative Pruning**, **Structural Friction**, and the **Law of Entropic Proportionality**.\n\nThe repository now includes the complete results and publication artifacts for **Experiment Two: The Battle of Architectures**.\n\nThe foundational paper introduces Semantic Thermodynamics and the Narrative Gravity framework.\n\nExperiment Zero compared a high-entropy baseline against a semantically constrained request over 100 API calls.\n\n**Result:**\n\n**79.29% reduction in completion tokens****60.73% reduction in mean end-to-end latency**- identical material answer across both conditions\n- collapse from 50 distinct textual realizations to a single realization under the optimized condition\n\nThe experiment established that additional semantic structure can increase prompt length while dramatically reducing downstream generation.\n\n### Paper II — The Collapse of Phase Space: Narrative as an Operational Thermodynamic Quantity in Inference Pruning\n\nThe second paper investigates the **entropic gradient**.\n\nExperiment One progressively increased narrative constraint density and found that efficiency does not improve monotonically.\n\nFor a low-entropy extraction task, the optimum occurred under **Light Gravity**, not under maximum constraint.\n\nBeyond the optimum, redundant instructions, conflicting rules, auxiliary objectives, and output complexity produced **Structural Friction**.\n\nThis led to the operational formulation of the **Law of Entropic Proportionality**:\n\nOptimal structure is not maximal. It is proportional.\n\nThe paper formalizes Narrative Pruning as the reduction of effective output phase space before the executor materializes its response.\n\nThe third experiment moves Semantic Thermodynamics from prompt-level optimization to **system architecture**.\n\nInstead of comparing individual prompt formulations, Experiment Two compares two complete inference architectures.\n\nA single `gpt-4o-2024-11-20`\n\ninference receives:\n\n- a synthetic\n**8,192-token corporate System Prompt**; - a noisy user request;\n- a deliberately chaotic enterprise email thread.\n\nThe System Prompt simulates the accumulation of generic enterprise instructions around safety, compliance, privacy, legal caution, formatting, tone, auditability, and exception handling.\n\nIt is a synthetic experimental stressor and is **not presented as a reproduction or estimate of any company's proprietary production System Prompt**.\n\nA two-stage architecture:\n\n**1. Semantic Micro-Router**\n\n`gpt-4o-mini-2024-07-18`\n\nThe router receives only the noisy user request and compiles it into a compact Narrative Gravity formula containing:\n\n- Persona\n- Objective\n- Scope\n- Negative Constraints\n- Output Matrix\n\n**2. Executor**\n\n`gpt-4o-2024-11-20`\n\nThe executor receives only:\n\n- the dynamically generated semantic formula;\n- the source email thread.\n\nThe large model therefore begins inference inside a task-specific semantic field instead of carrying a large generic policy layer.\n\nThe final experiment completed **50 preregistered paired observations**.\n\nRoute A:\n\n**10,282.28 mean total tokens per pipeline**\n\nRoute B:\n\n**2,624.84 mean total tokens per pipeline**\n\n**Route B reduced total token consumption by 74.47%.**\n\nIt used fewer total tokens in **50/50 paired observations**.\n\nWith observed prompt caching:\n\n- Route A:\n**US$0.0136206 per extraction** - Route B:\n**US$0.0044209 per extraction**\n\n**Route B reduced cost by 67.54%.**\n\nWithout prompt caching:\n\n- Route A:\n**US$0.0260078** - Route B:\n**US$0.0054961**\n\n**Route B reduced cost by 78.87%.**\n\nThe dynamic architecture remained substantially cheaper even though it paid for an additional micro-router inference.\n\nNormalized material accuracy:\n\n- Route A:\n**36/50 — 72%** - Route B:\n**50/50 — 100%**\n\nRoute A produced **14 material extraction failures**.\n\nRoute B produced **zero**.\n\nDynamic routing therefore improved cost and quality simultaneously.\n\nRoute A retained the expected one-hop advantage:\n\n- Route A:\n**2.257 s mean service latency** - Route B:\n**3.246 s**\n\nRoute B performs two sequential inferences and therefore pays an additional service-latency cost.\n\nHowever, its p95 service latency was lower:\n\n- Route A:\n**6.383 s** - Route B:\n**4.359 s**\n\nUnder the project's actual token-rate constraints:\n\n- Route A:\n**24.202 s mean operational wall time** - Route B:\n**3.284 s**\n\n**Route B reduced operational wall time by 86.43%.**\n\nThe difference emerged because the large static request repeatedly approached the project's token-per-minute envelope and required admission control.\n\nExperiment Two produced an important result before the final controlled run even began.\n\nThe original architecture launched both routes without rate-aware admission control.\n\nAcross 150 attempted API calls:\n\n**69 calls failed with HTTP 429**- Route A completed only\n**3/50** final calls - Route B's micro-router completed\n**50/50** - Route B's executor completed\n**28/50** - only\n**2/50 complete A/B pairs** survived\n\nRoute A injected approximately **10K prompt tokens in a single atomic request**, rapidly exhausting the available token-per-minute capacity.\n\nThe final experiment therefore introduced a rate-limit governor that converted rejection into explicit admission waiting.\n\nThis allowed all 50 pairs to complete while preserving the infrastructure burden as a measured variable.\n\nThe governor did not make the static architecture efficient.\n\n**It prevented it from collapsing.**\n\nRoute A achieved an exceptional **96.756% prompt-cache ratio**.\n\nOnly 332.24 of its 10,242 average prompt tokens were uncached.\n\nYet Route A remained **3.08× more expensive** than Route B.\n\nUnder a no-cache counterfactual, it became **4.73× more expensive**.\n\nCaching discounts repeated input.\n\nIt does not remove that input from the model's semantic field.\n\nThis distinction is central to Semantic Thermodynamics: reducing the price of context is not equivalent to reducing the inference space created by that context.\n\nExperiment Two also exposed a clean architectural boundary.\n\nRoute B achieved:\n\n**50/50 normalized material accuracy**\n\nbut only:\n\n**3/50 native JSON adherence**\n\nMost remaining failures consisted solely of Markdown code fences around otherwise correct JSON.\n\nThis separates two different engineering problems:\n\n**Semantic routing controls what the model should compute.**\n\n**Structured decoding controls how the result must be serialized.**\n\nThe next experimental stage combines:\n\nDynamic Semantic Routing + Structured Decoding\n\nrather than forcing natural-language instructions to perform both functions.\n\nThis release includes the complete **Experiment Two publication bundle**.\n\nThe ZIP archive contains the experimental provenance required to audit and reproduce the run, including:\n\n- experiment source code\n- dependency specification\n- README\n- methodology\n- preregistration\n- data dictionary\n- source documentation\n- exact chaotic email thread\n- exact chaotic user request\n- exact 8,192-token synthetic System Prompt\n- exact micro-router System Prompt\n- gold standard\n- experiment configuration\n- request architecture\n- environment metadata\n- pre-run manifest\n- raw API attempt journal\n- rate-limit event log\n- all generated router formulas\n- all final model outputs\n- primary results CSV\n- paired-results CSV\n- aggregate metrics\n- machine-readable analysis summary\n- publication figures\n- complete Markdown experiment report\n- SHA-256 provenance manifest\n\nThe publication bundle is intended to make the experiment **inspectable, reproducible, and falsifiable**.\n\nIf you do not want to unpack the full publication bundle, start with:\n\n`experiment_two_results.csv`\n\nThe 100 final observations: 50 Route A and 50 Route B.\n\n`paired_results.csv`\n\nThe 50 direct paired comparisons and B−A deltas.\n\n`god_prompt.txt`\n\nThe exact synthetic 8,192-token System Prompt used by Route A.\n\n`experiment_two.py`\n\nThe complete experimental implementation.\n\n`experiment_two_report.md`\n\nThe automatically generated analysis of the completed run.\n\nFor complete provenance, use the ZIP archive.\n\nA suggested repository layout is:\n\n```\nsemantic-thermodynamics/\n│\n├── README.md\n│\n├── papers/\n│   ├── semantic_thermodynamics.pdf\n│   ├── collapse_of_phase_space.pdf\n│   └── battle_of_architectures.pdf\n│\n├── experiment_zero/\n│   ├── experiment_zero.py\n│   └── experiment_zero_results.csv\n│\n├── experiment_one/\n│   ├── experiment_one.py\n│   └── experiment_one_results.csv\n│\n└── experiment_two/\n    ├── experiment_two.py\n    ├── experiment_two_results.csv\n    ├── paired_results.csv\n    ├── god_prompt.txt\n    ├── experiment_two_report.md\n    └── experiment_two_publication_bundle.zip\n```\n\nThe ZIP contains the complete Experiment Two directory tree and should be treated as the canonical reproducibility package for that experiment.\n\nInstall the dependencies:\n\n```\npip install -r requirements.txt\n```\n\nSet your OpenAI API key:\n\n```\nexport OPENAI_API_KEY=\"...\"\n```\n\nThen execute the experiment according to the instructions included in the publication bundle.\n\nRunning the full benchmark performs paid API calls.\n\nRate limits, model availability, pricing, prompt caching behavior, and infrastructure conditions may differ across accounts and execution dates. The exact model snapshots and experiment configuration used for the published run are preserved in the artifacts.\n\nThe English publication run completed:\n\n**50/50 measured pairs****100 final observations****150 measured model calls****6 warm-up calls****0 HTTP 429 responses during the governed run****0 retries****36/36 publication hashes validated**\n\nTwo successful Route B executor calls were absent from the append-only raw-attempt journal. Their request IDs, outputs, token telemetry, costs, latencies, and hashes remained preserved and independently cross-validated in the primary artifacts.\n\nNo missing raw headers were reconstructed.\n\nThe 100 final observations and all paired experimental results remain complete.\n\nAcross the three experiments, the same engineering principle has now appeared at three different levels:\n\n**Experiment Zero:** semantic constraint can reduce inference output and latency.\n\n**Experiment One:** constraint has an optimum; excessive constraint creates Structural Friction.\n\n**Experiment Two:** dynamically generating the appropriate constraint field can outperform carrying a large static instruction field through every inference.\n\nSemantic Thermodynamics therefore moves the optimization target away from prompt length alone.\n\nThe relevant variable is the relationship between:\n\n**task entropy → constraint density → inference behavior → infrastructure cost**\n\nOr, more simply:\n\nThe cheapest token is not merely the token you avoid sending. It is the branch of inference you prevent from becoming computationally relevant.\n\n**Tauan Vinicius Guahyba Sloboda**\n\nSemantic Thermodynamics Research Series", "url": "https://wpnews.pro/news/two-llm-calls-beat-one-67-cheaper-and-100-vs-72-extraction-accuracy", "canonical_source": "https://github.com/tauansloboda/semantic-thermodynamics", "published_at": "2026-08-12 02:21:59+00:00", "updated_at": "2026-08-12 02:40:44.942734+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research"], "entities": ["OpenAI", "GPT-4o", "GPT-4o-mini", "Semantic Thermodynamics"], "alternates": {"html": "https://wpnews.pro/news/two-llm-calls-beat-one-67-cheaper-and-100-vs-72-extraction-accuracy", "markdown": "https://wpnews.pro/news/two-llm-calls-beat-one-67-cheaper-and-100-vs-72-extraction-accuracy.md", "text": "https://wpnews.pro/news/two-llm-calls-beat-one-67-cheaper-and-100-vs-72-extraction-accuracy.txt", "jsonld": "https://wpnews.pro/news/two-llm-calls-beat-one-67-cheaper-and-100-vs-72-extraction-accuracy.jsonld"}}