I ran 19 tasks through Kimi K3.1 Pro and Claude Fable 5.2. The gap wasn't capability.

A developer benchmarked Kimi K3.1 Pro and Claude Fable 5.2 across 19 script-graded tasks run through the same OpenAI-compatible endpoint, finding the two models one task apart (17/19 vs 18/19) and both misses attributable to slips rather than capability limits. The runs showed 84% of Kimi's output tokens were thinking tokens, making it roughly $63.3 per million visible tokens versus $40.0 for Fable despite Fable's higher list price, and surfaced portability pitfalls including non-standard temperature handling, 200 responses with empty bodies, and divergent structured-output and tool-calling behavior.

AI disclosure: parts of this post were drafted and edited with AI assistance. The tests, numbers and charts come from my own runs on 12 November 2026. Short version: for a user-facing chat product I'd pick Claude Fable 5.2. For batch work on a budget I'd pick Kimi K3.1 Pro. On capability the two are one task apart out of nineteen, and both misses were slips rather than limits. I ran all 19 tasks on the same day, through the same OpenAI-compatible endpoint, on the same machine. Only the model string changed. Code tasks got up to three repair rounds; everything else was one shot. Every answer was script-graded, so the model's explanation never earned points. base url = "https://api.crazyrouter.com/v1" So what actually separated them? | Dimension | Tasks | Kimi K3.1 Pro | Claude Fable 5.2 | |---|---|---|---| | Code repair | 3 | 3/3 | 3/3 | | From scratch | 3 | 3/3 | 3/3 | | Tool use | 3 | 2/3 | 2/3 | | Repo QA | 3 | 3/3 | 3/3 | | Instruction following | 3 | 2/3 | 3/3 | | Resistance to misleading | 2 | 2/2 | 2/2 | | Long context | 2 | 2/2 | 2/2 | | Total | 19 | 17/19 | 18/19 | All six code tasks passed in the first round for both models. Repo QA and long context were clean. On the fake-model question, neither one invented a context window or a price. The SQL task. "Total order amount last month." Kimi called search docs and searched documentation. Fable called run sql , but the query hit information schema instead of the orders table. Both failed, differently. The writing task. Kimi overshot the 140–160 character range this time. The same task passed in an earlier dry run at 155 characters, so I treat it as output instability, not a capability gap. Neither failure produces an error in production. Both produce rework. Where does the money actually go? | Model | Output tokens | Of which thinking | Visible text | |---|---|---|---| | Kimi K3.1 Pro | 10,896 | 9,175 | 1,721 | | Claude Fable 5.2 | 6,386 | 0 | 6,386 | 84% of Kimi's output is thinking. That is where its cost lives. Fable has no thinking tokens, so everything you pay for is readable. Recompute per visible token and the ranking flips: roughly $63.3 per million visible tokens for Kimi, $40.0 for Fable. The model with the 4x list price is cheaper per readable token. The log task hid a key at line 287; the module task put LAST RETRY = 43 near the end. Both models answered both, with a handful of output tokens. But the inputs were about 14K tokens. Kimi publishes a 1,048,576-token window; Anthropic's product page for Fable 5.2 does not publish a number. So this proves only that at 14K tokens neither model loses the needle. It says nothing about million-token capacity. To test that, build an input near the limit — I did not. Temperature is not portable. Kimi returns 400 for temperature: 0.7 , and the error does not name the field. gpt-6.1-terra requires exactly 1. Claude accepts 2 and silently ignores it. Gemini returns 429 for 5, which looks like a rate limit. DeepSeek returns a clear 400. Six models, five behaviors. A 200 can carry an empty body. With a 1,200-token cap, Kimi returned 200, finish reason: length , zero characters, and 1,197 thinking tokens. Validate content length and finish reason, not the status code. Structured output has dialects. Claude routes reject json object with a 400 and point to json schema ; that works. The others accept json object . Tool calling is not universal. gpt-6.1-terra returned "service temporarily unavailable" for tool calls three times. Gemini needs tool choice: required ; with auto , four of five runs came back empty with content filter . So what would I pick? | Workload | Pick | Why | |---|---|---| | Customer-facing chat | Fable 5.2 | 130.1s vs 401.6s wall clock across the suite | | Batch / async | Kimi K3.1 Pro | Batch inference bills at 60% of list | | Output parsed by code | Either, plus a validator | JSON and tables passed for both; Kimi missed a character range once | | Long-context retrieval | Either at 14K tokens | Ceiling untested; measure with your own material | | Tool orchestration | Either, with a schema check | Both missed the same routing question | Kimi: 27,131 input and 10,896 output, $0.163. Fable: 36,074 input and 6,386 output, $0.544. That is 3.3x, not 4x, because the token shapes pull in opposite directions. Three workloads at list price: a 100k-round support bot around $460 vs $1,840; a thousand 8k/1.5k documents around $31 vs $124; a coding assistant at 2M in / 400k out per developer around $8 vs $32. Current availability is on the model catalog https://crazyrouter.com/en/models , which is what I checked before the run. If your input/output ratio differs from mine, the pricing page https://crazyrouter.com/en/pricing?utm source=devto&utm medium=article&utm campaign=offsite article&utm content=cr 1e98c0ee00ea mid text is where to redo the arithmetic. 19 tasks, two models, one day. Enough for a direction, not a verdict. Not tested: longer contexts, multi-turn tool use, or pricing beyond the $8/$40 list and the $0.20 cache read. Three steps for your own numbers: ask your most common question ten times and record the median and the worst case; have the model write one real draft under your platform's rules and count your edits; run two max tokens settings three times each and count the length finishes. Twenty minutes. Your numbers will beat this post.