{"slug": "context-graph-makes-haiku-5-5-23-faster-for-19-less-money-in-fewer-steps", "title": "Context Graph Makes Haiku 5.5 23% faster, for 19% less money, in fewer steps", "summary": "Giving Claude Haiku 5.5 a single upfront query to the Context Graph API made the model answer 17 engineering questions about a large codebase 23% faster on average, for 19% less money and in 17% fewer steps, according to a Postman benchmark of 102 graded runs (17 questions, 3 runs per arm, 51 runs per arm). Per prompt, cost fell from $0.273 to $0.221, input tokens dropped 20% from 459,000 to 365,000, output tokens fell 11% from 17,300 to 15,400, and average time to answer dropped from 19.6 minutes to 15.2 minutes even including the graph query, while time Haiku spent working fell 57% from 19.6 to 8.5 minutes. Accuracy on the 16 questions with answer keys moved from 0.623 to 0.638, a +0.015 gain the author called small, and the model matched or beat its own accuracy on every cross-repository dependency question.", "body_md": "# Context Graph Makes Haiku 5.5 23% faster, for 19% less money, in fewer steps\n\nWe gave [Claude Haiku 5.5](https://www.anthropic.com/claude/haiku) the same 17 engineering questions about a large codebase twice: once on its own, and once with a single query to the [Context Graph API](https://blog.postman.com/introducing-the-context-graph-api-one-map-of-your-api-ecosystem/) up front. With the graph, it answered 23% faster on average, for 19% less money, in fewer steps. It also matched or beat its own accuracy on every cross-repository dependency question we asked.\n\nHaiku 5.5 is already a small, fast, cheap model. The question was whether the Context Graph makes it more useful per prompt: the same quality of answer, with less time, less money, and less wandering through the code. Across 102 graded runs, it does.\n\n## The headline numbers\n\nEverything here is per prompt, averaged over 51 runs per arm (17 questions, 3 runs each).\n\n|  | Haiku 5.5 alone | Haiku 5.5 + Context Graph | Change | \n|---|---|---|---|\n| Cost | $0.273 | $0.221 | 19% less | \n| Input tokens | 459,000 | 365,000 | 20% fewer | \n| Output tokens | 17,300 | 15,400 | 11% fewer | \n| Time to answer, including the graph query (average) | 19.6 min | 15.2 min | 23% faster | \n| Time Haiku spends working (average) | 19.6 min | 8.5 min | 57% less | \n| Steps per question | 10.2 | 8.5 | 17% fewer | \n| Tool calls per question | 32.3 | 29.5 | 9% fewer | \n| Accuracy (the 16 questions with an answer key) | 0.623 | 0.638 | +0.015 | \n\nI ran the same 17 questions through Haiku 5.5 on its own and with the Context Graph, three times each. The clearest result is that the graph made Haiku do less work. Per question, it used 20% fewer input tokens, made 9% fewer tool calls, and cost 19% less.\n\nThe timing is interesting because the graph query itself adds time. Haiku spent an average of 8.5 minutes working with the graph, compared with 19.6 minutes on its own. Even after including the graph query, the total time to answer fell from 19.6 to 15.2 minutes. So the query took time, but Haiku saved more time than it spent waiting for it.\n\nAccuracy moved from 0.623 to 0.638 on the 16 questions with answer keys. That’s a small improvement, and I wouldn’t make a big claim from that difference alone. What these runs show more clearly is that Haiku reached roughly the same accuracy with less searching, fewer tokens, lower cost, and a faster answer.\n\nThe difference becomes easier to understand when you look at where Haiku starts its search.\n\n## The simplest way to think about it\n\nPicture someone dumping a huge bin of LEGOs on the table and asking which of these pieces snap onto this one blue brick. With only the bin, you have one move: pick up bricks one at a time, try each against the blue one, and set aside the ones that fit. A faster, more experienced builder gets through the bin quicker. They’re still going through the whole bin, though, and they’ll never find a piece that isn’t in it, like one sitting in a box in the other room.\n\nNow hand them the instruction booklet. It has a parts list that says which pieces connect to which, and which box each one lives in, including boxes that aren’t on your table. The builder doesn’t try every brick anymore. They start from the list, grab what it names, and snap the pieces on to make sure they fit.\n\nHaiku 5.5 is the builder, the checked-out repositories are the bin, and the Context Graph is the booklet. The graph is an index of services, endpoints, schemas, and the relationships between them across everything connected to it. (For the longer argument, see [why AI coding agents need context graphs](https://blog.postman.com/why-ai-coding-agents-need-context-graphs/).) It doesn’t replace the building. Haiku still has to confirm the pieces fit, which means reading code. The graph tells it where to start.\n\n## What we tested: 17 questions, 802 repositories, 102 runs\n\nBoth arms get the same model (Claude Haiku 5.5), the same read-only tools (read file, list directory, grep, glob, `git log`, [` git blame`](https://git-scm.com/docs/git-blame)), the same questions, and the same code at one pinned commit. The code is one public GitHub organization with 802 repositories.\n\nThere’s exactly one difference. The graph arm queries the Context Graph once before Haiku starts, and the results go in front of Haiku as a set of candidates to verify. The baseline never hears about the graph. The graph query is the same short question for every run of a given prompt.\n\nEach question runs 3 times per arm, because one run of a language model is a sample, not a measurement. 17 questions × 3 runs × 2 arms is 102 runs.\n\n### The 17 prompts\n\nThese are plain prompts, the kind an engineer would ask. None of them tells Haiku where to look or how many answers to find.\n\nCross-repository dependencies (9 questions):\n\n| Question | What it asks | \n|---|---|\n| Integrations (7) | Identify every repository that integrates with `X` , with a file and line proving it. | \n| Message consumers | Which repositories consume a [Kafka](https://kafka.apache.org/documentation/) topic that one named service publishes to? | \n| Most depended-on | Which single external service do the most repositories depend on, and which repositories are they? | \n\nInventory questions (6 questions):\n\n| Question | What it asks | \n|---|---|\n| HTTP endpoints | Which services serve HTTP endpoints, and on what paths? | \n| Message-bus consumers | Which services consume from a message bus, and which topics? | \n| Runtime calls | Which services call another service at run time? | \n| Deployed services | Which services do we actually deploy, and how? | \n| Schemas | Which services declare a machine-readable data or API contract? | \n| Telemetry | Which services emit traces, metrics, or errors about themselves? | \n\nImpact questions (2 questions):\n\n| Question | What it asks | \n|---|---|\n| API callers | Which services call the organization’s own HTTP API? | \n| Deployment blast radius | A breaking change is coming to an SDK we depend on. Which deployed services use it and would need redeploying? | \n\n## How we graded: 70% answer key, 5% citations, 25% judge\n\nEvery answer gets graded three ways, and the weights are fixed in the benchmark: 70% against an answer key, 5% on citations, and 25% on a judge.\n\n### The answer key (70%)\n\nEvery question has a list of the repositories that correctly answer it, and that list comes from the code. We search the checked-out repos for the thing the question asks about. It never comes from the Context Graph. Grading the graph against its own output would be marking its own homework.\n\nThe score on the key is recall multiplied by a precision factor. [Recall and precision](https://en.wikipedia.org/wiki/Precision_and_recall) measure different things here: recall is how much of the key the answer found, and the precision factor knocks the score down if the answer also names a pile of repos that aren’t in the key. A shotgun list doesn’t win. Being right does.\n\nWe widened the keys, but carefully. For every repo the graph named that the key was missing, we opened the code and checked it against the question’s definition. A repo only joined the key if the code on disk showed it, with a file and line as evidence. That added 14 endpoint services, 16 deployed services, 9 telemetry emitters, 8 runtime callers, 2 schema declarers, and 1 consumer. The graph proposes, and the code decides.\n\n### Citations (5%)\n\nEvery file path the answer cites gets checked against the repo at the pinned commit. Invent a path and you get caught.\n\n### A judge (25%)\n\nA language model (Claude Opus 4.7) scores each answer against a rubric. It gets the smallest weight because it can’t inspect the repo the way the other two graders can.\n\nA run passes at 0.5 or higher. Name matching is forgiving: `_` and `-` are treated the same, GitHub URLs count, and a component like `ingester-service` gets credited to its parent repo.\n\n### What counts as the graph arm’s answer\n\nFor the answer-key part of the score, we score the graph arm on what the graph returned plus what Haiku committed to, since both are part of what the system hands the user. Citations and the judge only look at Haiku’s answer. We also report Haiku’s committed list on its own, so you can see how much of any score comes from the graph and how much from Haiku. The baseline has no graph output, so its score is its committed answer.\n\n## Results: 19% less money, 23% less time\n\n### Haiku 5.5 is 19% cheaper with the Context Graph\n\nHere’s the bill for 51 runs per arm, at Haiku 5.5’s published rate of $0.50 per million input tokens and $2.50 per million output tokens (see [Anthropic’s pricing](https://www.anthropic.com/pricing)). That’s the higher, over-100K-token tier, so treat these as upper bounds.\n\n| 51 runs per arm | Haiku alone | Haiku + Context Graph | Saved | \n|---|---|---|---|\n| Input tokens | 23.41 million | 18.61 million | 4.80 million (20%) | \n| Input cost | $11.70 | $9.31 | $2.39 | \n| Output tokens | 880,000 | 787,000 | 93,000 (11%) | \n| Output cost | $2.20 | $1.97 | $0.23 | \n| **Total cost** | **$13.90** | **$11.27** | **$2.63 (19%)** | \n| Per question | $0.273 | $0.221 | $0.052 | \n| Per 1,000 questions | $273 | $221 | $52 | \n\nIt comes down to tokens. In an agent loop every step resends the whole transcript, so input tokens make up about 84% of the bill, and the graph arm used 365,000 input tokens per question against 459,000. These figures count model tokens only, with no charge for the graph query itself.\n\nBy question type, the dependency questions got 21% cheaper ($0.376 to $0.298 per question) and the inventory and impact questions got 13% cheaper ($0.156 to $0.135). The graph arm was cheaper on 12 of the 17 questions, by as much as 48% (Vercel integrations, $0.365 to $0.191) and 44% (consumers of a topic, $0.732 to $0.407). It cost more on 5, mostly questions that were cheap to begin with. Runtime calls went from $0.061 to $0.099 and HTTP endpoints from $0.073 to $0.119, because the graph hands Haiku a long list to chew through.\n\n### Haiku 5.5 answers 23% faster with the Context Graph\n\nTime is trickier, because there are two honest ways to measure it, and they tell different parts of the story.\n\n| Minutes per question | Haiku alone | Haiku + Context Graph | \n|---|---|---|\n| Haiku’s own working time, average | 19.6 | 8.5 | \n\nHaiku’s own working time dropped 57% on average, from 19.6 minutes to 8.5. That’s the time spent reasoning and searching, with the wait for the graph left out.\n\nEnd to end, including the wait for the graph, a question took 15.2 minutes instead of 19.6. That’s 23% faster on average. Even after paying for the graph query on every question, the graph arm came out 3.8 hours ahead across the 51 runs.\n\n### Haiku 5.5 makes 9% fewer tool calls with the Context Graph\n\n| Per question | Alone | + Graph | Change | \n|---|---|---|---|\n| Steps | 10.2 | 8.5 | -17% | \n| Tool calls | 32.3 | 29.5 | -9% | \n| `read_file` | 8.8 | 6.3 | -28% | \n| Whole-codebase `grep` | 9.6 | 7.6 | -21% | \n\nBy question type, the dependency questions went from 35.6 to 32.0 tool calls (10% fewer) and the inventory and impact questions from 28.5 to 26.8 (6% fewer). The graph arm made fewer calls on 9 of the 17 questions. The biggest drops were schemas (33.0 to 17.7), telemetry (24.3 to 11.3), and deployed services (34.3 to 20.7).\n\nThat reduction is on purpose. The graph arm is told the candidates come from an index, so it doesn’t need to re-confirm what the index already says. That’s what you want from a context layer: Haiku spends its steps on what’s missing from the index instead of rediscovering what’s in it.\n\n### It reaches beyond the checkout: 49 repositories named per question, up from 29\n\nThe context graph arm names a lot more repos in its answers: 49 per question against 29. Some of those have no code in the checked-out repositories at all. They’re external systems called at run time, or services living elsewhere in the wider ecosystem. You can’t grep your way to those. The score can’t give credit for them either, since the answer keys come from the checked-out code, so every number in this post undercounts this benefit.\n\n### It holds answers Haiku doesn’t always use: 64% of the telemetry key arrives, 13% gets reported\n\nOn telemetry and runtime-call questions, the graph’s own rows covered 64% and 55% of the answer key. Haiku on its own covered 8% and 6%. With the graph, Haiku committed only 13% and 3% of the key in its final list, so it left a lot of what the graph found on the table. That’s Haiku’s call about what to report, not a limit of the graph. It also points at the clearest upside left: the information is already arriving, and using it better is the next gain.\n\n```\nShare of the answer key covered (telemetry)\n  Haiku alone                     ██░░░░░░░░░░░░░░░░░░░░░░░░░░░░   8%\n  Haiku's final list, with graph  ████░░░░░░░░░░░░░░░░░░░░░░░░░░  13%\n  Haiku + graph, everything       ███████████████████░░░░░░░░░░░  64%\n```\n\n## Where the Context Graph earns its place: 19% cheaper, 23% faster, 9 of 9 ahead on dependencies\n\nFor a small, fast model, the graph isn’t about making each answer smarter. It’s about making each answer cheaper to get to: fewer tokens, fewer steps, less time, and the same or better accuracy. That worked out to 19% less money, 20% fewer input tokens, and 57% less work from Haiku itself.\n\nIt earns the most on the question every engineer dreads: what else depends on this? When the answer is scattered across dozens of repos with no file that lists them, Haiku on its own has to find each one by hand. With the parts list in hand, it started from the answer and came out ahead on all 9 dependency questions.\n\nThe gaps are real too. The graph should help Haiku report more of what it already receives, and the inventory questions need work before they show the graph’s value in a score. On cost, speed, and cross-repo reach, though, the result held up across 102 graded runs. Haiku 5.5 with the Context Graph does the same job for less, and on the hardest dependency questions it does it better.\n\n## Try it on your own code\n\nPick a cross-repo question your team keeps asking, like what else calls this service. Run it once with a plain agent, and once with a Context Graph query in front of it. Compare the time, the tokens, and the list of repos each run returns, and check any repo that only one of them named. Start from the [Context Graph API workspace](https://www.postman.com/postman/workspace/postman-context-graph-api), or point an MCP client at the graph by following the [MCP query docs](https://learning.postman.com/docs/use/context-graph/mcp-query).", "url": "https://wpnews.pro/news/context-graph-makes-haiku-5-5-23-faster-for-19-less-money-in-fewer-steps", "canonical_source": "https://blog.postman.com/context-graph-makes-haiku-5-5-23-faster-for-19-less-money-in-fewer-steps/", "published_at": "2026-10-19 15:00:01+00:00", "updated_at": "2026-10-19 15:15:10.824430+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "ai-search"], "entities": ["Claude Haiku 5.5", "Context Graph API", "Postman", "Anthropic"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/context-graph-makes-haiku-5-5-23-faster-for-19-less-money-in-fewer-steps", "markdown": "https://wpnews.pro/news/context-graph-makes-haiku-5-5-23-faster-for-19-less-money-in-fewer-steps.md", "text": "https://wpnews.pro/news/context-graph-makes-haiku-5-5-23-faster-for-19-less-money-in-fewer-steps.txt", "jsonld": "https://wpnews.pro/news/context-graph-makes-haiku-5-5-23-faster-for-19-less-money-in-fewer-steps.jsonld"}}