{"slug": "i-built-the-same-agent-in-strands-langgraph-and-crewai-and-recorded-every-llm-to", "title": "I built the same agent in Strands, LangGraph, and CrewAI — and recorded every LLM call to see how they actually differ", "summary": "A developer built the same tech-news digest agent in three frameworks — Strands, LangGraph, and CrewAI — and routed all 27 executions through a local recorder proxy to compare LLM calls directly. LangGraph's explicit verify/revise loop cut output variance by 77% (word-count spread 13 to 3) but at 2.5x the tokens and 2.5x the latency, while CrewAI produced byte-identical output across runs with a fixed four-call structure and Strands self-corrected adaptively with variable call counts. Dependency footprints also diverged: Strands 262MB (81 packages), LangGraph 71MB (45), and CrewAI 699MB (142).", "body_md": "Which framework should I actually use? Every comparison article has an\n\nopinion. Almost none of them has data.\n\nSo I built the same agent three times — once in **Strands**, once in\n\n**LangGraph**, once in **CrewAI** — ran 27 executions, and routed every\n\nsingle LLM call through a local recorder proxy so the logs are directly\n\ncomparable. The frameworks write different log formats, different trace\n\nshapes, different everything. A one-file proxy in front of all of them fixed\n\nthat.\n\nThe headline finding: LangGraph's explicit verify/revise loop cut output\n\nvariance by **77%** (word-count spread 13 -> 3) — at 2.5x the tokens and 2.5x\n\nthe latency. Explicit control buys determinism, and you pay for it in tokens\n\nand latency.\n\n[https://github.com/sunnydachs/agent-framework-showdown](https://github.com/sunnydachs/agent-framework-showdown)\n\nA tech-news digest agent, identical across all three frameworks:\n\n`fetch_headlines` tool`word_count` tool, revising if out of band\nThe tools are deterministic and local — no network, no LLM inside them —\n\nbecause the thing being measured is the framework's behavior, not the tool's.\n\nSame model behind the proxy for all three runs.\n\nThe philosophy split is real, and it shows up in the implementation:\n\nThe dependency weight differs too: Strands 262MB (81 packages), LangGraph\n\n71MB (45), CrewAI 699MB (142). CrewAI's weight is the flip side of\n\n\"fastest to prototype.\"\n\nOne recorder proxy sits in front of every framework\n\n(`proxy/rec_proxy.py`): an HTTP server that forwards to any\n\nOpenAI-compatible endpoint and writes every request/response pair as JSONL —\n\nthe full messages the framework sent (system prompt, tool schemas,\n\nconversation history), the raw response (SSE-streamed or JSON), token usage,\n\nlatency, and status. The API key is stripped before writing.\n\nAn `X-Run-Label` header splits traces per run. Strands and LangChain stream\n\nSSE, so a small reassembler (`parse_sse.py`) normalizes streamed responses\n\ninto the same shape as non-streamed ones (content / reasoning / tool_calls /\n\nusage).\n\nSame trace shape across frameworks is what lets you diff messages, tokens,\n\nand latency exactly. Without it, \"which framework is slower\" is vibes.\n\nThree scenarios:\n\n`text` -> `content`),\ntesting how each framework copes with schema change\nWord-count spread across 3 runs (max - min, lower = more stable):\n\n| Scenario | Strands | LangGraph | CrewAI | \n|---|---|---|---|\n| base | 15 | **13** | **0** | \n| tight | 11 | **3** | 2 | \n| drift | 12 | 16 | 0 | \n\n|  | Strands | LangGraph | CrewAI | \n|---|---|---|---|\n| LLM calls | 3-5 (adaptive) | 1 (base) / 2 (tight) | 4 fixed | \n| Total tokens (base) | 2,370 | 2,007 | 2,532 | \n| Total latency | 4.2s | 4.7s | 3.8s | \n\n**LangGraph** in base mode made a single LLM call and wrote the draft in one\n\nshot. Word-count spread: 13 words. Switch to tight, and the explicit\n\nverify/revise loop fires: spread drops to 3 (−77%), tokens go 2,007 -> 5,262,\n\nlatency 4.7s -> 11.8s. The graph structure *guarantees* the loop runs — that\n\nis the product you're buying, and the price is 2.5x on everything.\n\n**CrewAI** produced byte-identical output across all 3 base and drift runs\n\n(96/96/96 words). temperature=0 plus the role prompt dominates. The flip\n\nside: the call structure is always 4 calls, fixed. The framework that \"just\n\nships it\" also just repeats it.\n\n**Strands** self-corrected inside its own loop. In one tight run the model\n\nwrote a 77-word draft, called `check_word_count`, reasoned \"77 is under 95,\n\nI need to expand it,\" revised itself to 103, and verified again — five LLM\n\ncalls, variable depth per run ([5, 3, 4]). The model deciding when to stop\n\nis part of the design, and the call count being adaptive is what you get for\n\ntrusting it.\n\nThe rename was absorbed 100%: all three frameworks' models called the tool\n\nwith the **new** argument name, zero wrong-arg calls, no error recovery\n\ntriggered. Worth being precise about what this does and doesn't show: a\n\none-argument rename is the gentlest possible schema change. Type changes,\n\nremoved arguments, or changed return shapes would likely break the\n\nmodel-driven side (which trusts its prompt over the wire), while LangGraph\n\nwould be immune — its tool calls live in code, not in a prompt.\n\n```\npython3 runs/run_matrix.py       # 27 runs in ~270s\npython3 runs/analyze_matrix.py   # -> artifacts/matrix_report.json\n```\n\nThe repo README has the full setup (three venvs, the recorder proxy, the\n\nper-framework run commands). One proxy in front of everyone is the whole\n\ntrick — when the frameworks write different log formats, a same-shape\n\nrecorder is the simplest honest answer.", "url": "https://wpnews.pro/news/i-built-the-same-agent-in-strands-langgraph-and-crewai-and-recorded-every-llm-to", "canonical_source": "https://dev.to/sunnydachs/i-built-the-same-agent-in-strands-langgraph-and-crewai-and-recorded-every-llm-call-to-see-how-4ae8", "published_at": "2026-09-21 05:47:42+00:00", "updated_at": "2026-09-21 05:52:55.914929+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "large-language-models"], "entities": ["Strands", "LangGraph", "CrewAI", "sunnydachs"], "alternates": {"html": "https://wpnews.pro/news/i-built-the-same-agent-in-strands-langgraph-and-crewai-and-recorded-every-llm-to", "markdown": "https://wpnews.pro/news/i-built-the-same-agent-in-strands-langgraph-and-crewai-and-recorded-every-llm-to.md", "text": "https://wpnews.pro/news/i-built-the-same-agent-in-strands-langgraph-and-crewai-and-recorded-every-llm-to.txt", "jsonld": "https://wpnews.pro/news/i-built-the-same-agent-in-strands-langgraph-and-crewai-and-recorded-every-llm-to.jsonld"}}