# Five agent engineering problems, with the numbers behind them > Source: > Published: 2026-08-19 19:52:00+00:00 The agent conversation on Reddit and in GitHub issues has moved. A year ago it was "what can agents do". Now it is "why does mine call the same tool nineteen times", and "what happens to my threads on August 26". Here are five problems that keep coming up, each with the specific fact I had to dig out to answer it. Every one of them has a number attached, because "it depends" is not an answer you can ship. `tool_choice` sticks The advice everyone gives is to set `max_iterations` . That caps your bill. It does not fix the bug. When you set `tool_choice` to `required` or to a named function, that setting **persists across model calls**. Your framework runs the tool, sends the result back, and the same forced setting rides along. The model is told again that it must call a tool. It obeys. That is the loop. The [merged fix in openai-agents-python](https://github.com/openai/openai-agents-python/pull/263) resets `tool_choice` to `auto` after tool execution. So step one is upgrade, not guardrail code. Step two is the part almost nobody does. A loop is not "many calls" — it is **the same call**. [Oracle's langchain-oracle patch](https://github.com/oracle/langchain-oracle/pull/50) detects repeats by matching the tool name *and identical arguments* in succession, with a `max_sequential_tool_calls` backstop defaulting to 8. Their earlier fix had set `tool_choice` to `none` after any tool result, which stopped loops and also broke a four-step diagnostic agent after its first call. An iteration counter cannot tell a six-step workflow from a six-step loop. Argument identity can. Full breakdown: [Your agent loops forever. It is probably tool_choice.](https://www.nihardaily.com/posts/your-agent-loops-forever-it-is-probably-toolchoice) If you are still on `/v1/threads` , you have days, not months. [OpenAI's deprecations page](https://developers.openai.com/api/docs/deprecations) sets removal at **August 26, 2026**, one year after the notice. No degraded mode, no grace period. The migration itself is small. Assistant becomes Prompt, Thread becomes Conversation, Run becomes Response, Run step becomes Item, and the create-then-poll loop collapses into a single `responses.create` call. The part that bites is one sentence in [the migration guide](https://developers.openai.com/api/docs/assistants/migration): OpenAI **will not provide an automated tool** for migrating Threads to Conversations. If your product shows chat history, that history is user-visible data sitting on someone else's server behind an endpoint that stops answering. Export the raw JSON this week even if you have not chosen a target format. Exported data can wait; deleted data cannot. Two things that surprised me: vector stores and files survive the cutover and just need `vector_store_ids` passed to the file search tool — but thread-created stores expire **seven days** after last use, so half of them are already gone. And Azure lands on the same date with a *different* destination: [Microsoft's docs](https://learn.microsoft.com/en-us/azure/foundry-classic/openai/how-to/file-search) point Azure users to Foundry Agents, not to the Responses API. Full breakdown: [The Assistants API shuts down August 26. Port it now.](https://www.nihardaily.com/posts/the-assistants-api-shuts-down-august-26-port-it-now) Straight from [the AutoGen README](https://github.com/microsoft/autogen): "AutoGen is now in maintenance mode. It will not receive new features or enhancements and is community managed going forward." Microsoft Agent Framework is the successor, built by the same teams as a merge of AutoGen and Semantic Kernel. Single agents port in an afternoon. Multi-agent teams do not, because AutoGen's event-driven `Team` becomes a typed, graph-based `Workflow` with edges you declare. You redraw the flow rather than translating it. Two changes throw no errors at all. `Agent` is stateless and keeps no history between calls, unlike `AssistantAgent` — your code runs, your agent forgets the last turn, and you attach `AgentSession` to fix it. And agents now keep calling tools until the job is done rather than stopping at a count you set, which is friendlier for simple tasks and harder to price. Check your model providers before planning anything. Microsoft's own pages disagree: [the migration guide](https://learn.microsoft.com/en-us/agent-framework/migration-guide/from-autogen/) lists Anthropic and Ollama clients as planned, while [the newer overview page](https://learn.microsoft.com/en-us/agent-framework/overview/) lists both as supported. Run a spike against your actual provider before you commit a sprint. Full breakdown: [AutoGen is in maintenance mode. Where to migrate now.](https://www.nihardaily.com/posts/autogen-is-in-maintenance-mode-where-to-migrate-now) Agent memory gets treated as a storage question — pick a vector store, summarize when the window fills. That misses the bill. Caching is a **prefix match**. Memory wants to live high in the prompt, next to the system instructions, which is the most expensive place in the request to edit. [Anthropic's caching reference](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) puts numbers on it: cache reads cost 0.1x base input, five-minute writes 1.25x, one-hour writes 2x. Swapping a read for a five-minute write is **12.5x the cost for the same tokens**. There is also a floor nobody mentions. The minimum cacheable prefix is 512 tokens on Opus 5, 1,024 on Sonnet 5, and 4,096 on Haiku 4.5. Fall short and nothing errors — the request just is not cached, and you learn about it from the invoice. That is exactly why routing cheap background summarization to a small model so often fails to help. The other useful distinction: clearing is not summarizing. [Context editing](https://platform.claude.com/docs/en/build-with-claude/context-editing) drops old tool results in order and swaps in placeholders, defaulting to a 100,000-token trigger while keeping the last three tool uses. It costs no extra model call and keeps the conversation shape. Compaction spends a call, rewrites history into prose, and permanently loses whatever the summarizer judged unimportant. For a tool-heavy agent, the bulk is old tool output — clear that first. Full breakdown: [Agent memory that does not wreck your prompt cache](https://www.nihardaily.com/posts/agent-memory-that-does-not-wreck-your-prompt-cache) [LangChain's State of Agent Engineering survey](https://www.langchain.com/state-of-agent-engineering) (1,340 responses, late 2025) found 89% of organizations have agent observability and 52.4% run offline evals. Most teams can watch the failure and cannot catch it first. The interesting part is *why* the teams with a gate still miss things. A [June 2026 paper on layer-isolated evaluation](https://arxiv.org/abs/2606.11686) broke one agent layer at a time and watched the metrics. The aggregate pass rate moved **1.7 to 5.9 percentage points**. The test slice matching the broken layer moved **25 to 91 points**. Read that as a release gate. Break your routing layer and an end-to-end suite reports a couple of points of noise — you would approve that ship. Sliced by layer, it screams. The cheap fix is that much of an agent is ordinary software. Routing rules, schema validation, escalation thresholds, memory writes: none of it is non-deterministic. The same paper's suite is 238 cases across 23 slices, 225 of them running in **2.39 seconds** with no model calls at all. Build that layer first and save the judge for what needs it. And if you do use a judge, calibrate per comparison. [A May 2026 study](https://arxiv.org/abs/2605.06939) shows that sharing one calibration across the models you are comparing can reverse the sign of the result — you ship the worse agent while the report says you shipped the better one. Full breakdown: [89% watch agents fail. Only half test before shipping.](https://www.nihardaily.com/posts/89-watch-agents-fail-only-half-test-before-shipping) Four of these five are the same shape: a default changed, or a setting persisted, and nothing threw an error. Sticky `tool_choice` , a stateless `Agent` , an uncached prefix, an averaged-away regression. Agents fail quietly far more often than they crash, which is why the fixes are mostly about making the failure visible rather than making the model smarter. If you have hit one of these and solved it differently, I would genuinely like to hear it — the argument-identity loop check in particular feels like something more frameworks should ship by default.