{"slug": "best-model-for-agentic-coding-october-2026-benchmarks", "title": "Best Model for Agentic Coding: October 2026 Benchmarks", "summary": "A developer's independent benchmark of 44 models on agentic coding tasks found no single dominant model, with GPT-6.1 Sol Pro topping the Gauntlet 7.2 board at 0.9574 and a router called Jev Router leading Forge 2.0 with a median of 0.7460. The two benchmarks, run through a fork of the open-source goose agent with no human in the loop and scored by automated test suites rather than model judges, measured how well models build a payments console and upgrade a Jira Forge app. The author noted that top performers mostly differed in tool-calling discipline—checking permissions, paginating correctly, and following through on stated next steps—and flagged GPT-6.1 Sol at $1.19 per build and MiMo V2.6 Flash at 31 cents per build as cost-effective options.", "body_md": "There's no single best model for agentic coding this month, at least on our two benchmarks. I spent the weekend reading the code behind the boards, and the scores matter less to me now than they did on Saturday. The models at the top weren't cleverer in any way I could see. Mostly they checked who was asking before showing an issue, kept reading a list until Jira said it was the last page, and when they wrote \"next\" they did it. If you pay for long agent runs, I'd look there before the price.\n\nPaying per token, I'd run GPT-6.1 Sol. It's 3rd on our Jira app benchmark, at $1.19 a build. On a tight budget, MiMo V2.6 Flash came 7th on the same benchmark for 31 cents a build. Scores run from 0 to 1.\n\nBoth benchmarks have a model build a real app through our fork of goose, the open-source coding agent, with nobody in the loop. Yes, a fork. We changed a lot of it for our own use. A scorer then runs the app. No model grades anything. [Gauntlet 7.2](https://leanzero.net/agentic-benchmarks?utm_source=devto&utm_medium=referral&utm_campaign=crosspost) asks for a payments console with a 3D view of the payments, with 150 model calls and one run per model. [Forge 2.0](https://leanzero.net/agentic-benchmarks/forge?utm_source=devto&utm_medium=referral&utm_campaign=crosspost) hands the model v1 of a Jira app built on Atlassian's Forge platform and asks it to ship v2, with no internet and 300 calls. Each cloud model gets two independent builds there, and the board ranks the median.\n\nThe Forge 2.0 app is a scope ledger for Jira sprints. It writes one row for each issue added to or removed from a sprint. Each row records which path recorded it first, a Jira event or the hourly reconcile job that re-reads Jira's history for missed events. The upgrade has to migrate v1's rows, and they must keep the label v1 gave them. The scorer installs each build on the same three Jira test sites, each seeded with different data, and checks them at seven points along the way. A critical defect, such as showing a person an issue they may not see, multiplies the score by 0.6.\n\nThese are goose runs with nobody watching. If a model ends a turn without calling a tool, goose stops, and so does Claude Code by default (a [Stop hook](https://code.claude.com/docs/en/hooks) can make it push on). It's what ended both of Haiku's builds. Each judgement here rests on one or two builds on our setup.\n\nThis closes off our runs from 2 to 19 October: 44 models, 43 entries on Gauntlet, 31 on Forge 2.0. I read both boards from our public dataset on Saturday 17 October, on Sunday 18 October at 19:34 EEST after DeepSeek's update came in, and again on Tuesday 20 October before this went out. Six models landed after the Sunday read: Pareto 26.10, Mistral Large 4, Nemotron 3.5 Super VL, Glimpse, Beam 1.0 and Muse Spark Open. Their places below are current, but I didn't read their builds for this piece, so none of the moments uses them. Each side is its own ranking, so read down a column. The last row skips down to Claude Haiku 5.5.\n\n| rank | Gauntlet 7.2 (one run each) | score | rank | Forge 2.0 (two builds each) | median | range | \n|---|---|---|---|---|---|---|\n| 1 | GPT-6.1 Sol Pro | 0.9574 | 1 | Jev Router (a router) | 0.7460 | 0.6398 to 0.8523 | \n| 2 | Qwen3.8 Max Prime | 0.8364 | 2 | Claude Opus 5.5 | 0.6745 | 0.6196 to 0.7294 | \n| 3 | Claude Sonnet 5.5 | 0.7984 | 3 | GPT-6.1 Sol | 0.5933 | 0.5371 to 0.6495 | \n| 4 | GPT-6.1 Sol | 0.7978 | 4 | Qwen3.8 Max Prime | 0.5765 | 0.5015 to 0.6515 | \n| 5 | Pareto 26.10 | 0.7971 | 5 | Claude Sonnet 5.5 | 0.5692 | 0.5485 to 0.5899 | \n| 6 | Pareto 26.10 Preview | 0.7957 | 6 | Gemini 4 Argon | 0.5434 | 0.2943 to 0.7926 | \n| 7 | Claude Opus 5.5 | 0.7926 | 7 | MiMo V2.6 Flash | 0.5283 | 0.5028 to 0.5538 | \n| 8 | MiMo V2.6 Flash | 0.7909 | 8 | Ember 1 | 0.5011 | 0.4710 to 0.5312 | \n| 9 | Muse Wonder | 0.7638 | 9 | DeepSeek V4.1 Flash 1017 | 0.4520 | 0.2422 to 0.6618 | \n| 10 | Gemini 4 Argon | 0.7078 | 10 | GLM-5.3 Prime | 0.4302 | 0.4046 to 0.4557 | \n| 11 | Jev Router | 0.6975 | 11 | Qwen3.8 Flash | 0.4077 | 0.3164 to 0.4990 | \n| 12 | DeepSeek V4.1 Flash 1017 | 0.6957 | 12 | Gemini 3.8 Flash | 0.3979 | 0.2999 to 0.4958 | \n| 19 | Claude Haiku 5.5 | 0.6610 | 29 | Claude Haiku 5.5 | 0.0621 | 0.0610 to 0.0632 | \n\nEmber 1 is Fireworks' model built on Kimi K3, and Jev Router is TypeSafe's router. OpenRouter's records showed 117 of Jev's 118 calls going to GPT-6.1 Sol (the [Jev piece](https://leanzero.net/blog/jev-router-benchmark-gpt-6-1-sol?utm_source=devto&utm_medium=referral&utm_campaign=crosspost) has the trace). We threw out nine invalid runs, and none of their scores are used here.\n\nTreat a Gauntlet gap under a few hundredths as a tie. Forge 2.0 splits models into comparison groups by goose build. Opus, Sol, MiMo and the September DeepSeek share one. Max Prime, Sonnet, Haiku and most others share another. Jev Router and Gemini 4 Argon ran a newer goose release. DeepSeek's update shares a group only with Mistral Large 4. Qwen3.8 Flash and Muse Wonder each have one of their own, and Glimpse, Muse Spark Open and Beam 1.0 share another. The board advises comparing within a group, and the local 27B is in none, so a rank across groups is my reading.\n\nGauntlet caps a run's score for each band of conditions it misses, so places 3 to 8 sit under one cap, 0.0075 apart, a tie on one run each. Qwen3.8 Max Prime, second, wrote in its closing note that \"every restart resumes, events stay contiguous\". The scorer killed its event relay halfway through a write, and the relay never picked back up.\n\nAll five moments below are from Forge 2.0, where I could open both builds of each model.\n\nClaude Haiku 5.5 is the surprise of the month, and its own words explain the score. Build 1 ended after 16 of its 300 calls with this, and no tool call after it. (A call is one model response, which can ask for several tools.)\n\n```\nI've written the v2 ledger layer and the numbers module. The sync path, migration,\nand resolvers are still to write, and I'm at about 18% of the context window.\nI'll get to them next.\n```\n\nBuild 2 lasted 37 calls and ended the same way: \"Given how much remains, I'll stop expanding the design here and finish the work in order, testing each part as I go.\" Build 1's manifest pointed at an admin page file it never wrote, so Forge's own lint failed, and the only time it had run lint was at about call 4, before it had written a single file. It never ran its own code.\n\nOpus 5.5 talks about its next step too. At message 331 of its first build it wrote \"Non-admins are refused. Next, a low-share run: does the backfill defer and resume correctly?\", and then made the tool call that runs it. Of its 40 text messages, only the final report ends a turn. And 53 of its 137 tool requests ran lint, invoked a function, fired a scheduled job or opened the page in a browser. GPT-6.1 Sol barely talks: three text messages in 47 calls. It writes files from inside shell commands, often running the checks in the same command, and ran 27 test, lint or run commands in that build.\n\nThe evidence is the tail, the call counts and the lint failure. Why Haiku stopped is my guess. Anthropic's own docs make it a fair one. Its [Haiku 5.5 prompting guide](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-haiku-5-5) says that \"with a long coding-agent system prompt at `low` effort, it sometimes stops early and hands the task back to the user\", and that moving from low to medium \"roughly halved early stopping\". We ran it at medium, so by Anthropic's own number it still stops early, about half as often as at low. The same guide offers a short piece of system-prompt text for this case. Goose's prompt doesn't carry it. The guide also says that at medium Haiku \"sometimes reports a code change as done without running a check\". The [Opus 5.5 guide](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-5-5) says its progress updates sometimes end the turn with text and no tool call, and warns that \"an unattended agent loop that treats such a turn as the end of the task stops running there\". Goose is that kind of loop.\n\nThe opposite failure burns the whole call budget. Grok 4.7's worse build and Hy4 Preview's used up all 300 calls. Grok tried `rg` 44 times on a machine without it and got \"command not found\" 42 times. Four of its replies hit the output limit repeating the same line or two, one of them 292,726 characters of \"The field values are present, so I'll run the full suite once more.\" That build ran at our medium pin, one level below xAI's default, and billed 1,452 reasoning tokens over the whole build. I can't tell the pin from the model there. Hy4 read the same starter file 45 times. Its prompt also shrank by more than half 15 times in that build, which may be why, and we haven't traced it.\n\nOur app shows a sprint's scope changes to anyone who opens the sprint, minus the issues that person may not browse. v1 already did this right: it asks Jira, as the person, which issues they can see. Two v2 builds leaked anyway, by different routes. Each took the 0.6 critical.\n\nQwen3.8 Omni Flash's worse build (0.1659) runs a request as the person only when it's passed `who: 'user'`. Its admin check passes it:\n\n```\n.request({ path: route`/rest/api/3/mypermissions?permissions=ADMINISTER`, who: 'user', method: 'GET' })\n```\n\nIts visibility lookup, under v1's comment, doesn't:\n\n```\n// Which of these issues can the invoking person browse, and under which key? Read as them: bulkfetch\n// leaves out issues the person may not see (§3). One request per 100 ids.\nexport async function visibleIssues(client, issueIds) {\n  ...\n    const page = await client.json({\n      path: route`/rest/api/3/issue/bulkfetch`,\n      method: 'POST',\n      body: { issueIdsOrKeys: ids.slice(i, i + 100), fields: ['summary'] },\n    });\n```\n\nSo all 62 of those lookups ran as the app, which can see everything, and my guess is a refactor that moved identity into a client object and missed one call, with the comment carried along. Our scorer plants this exact bug in a test copy of the app. Omni lost the same three checks the planted copy loses. Its other build asked as the person.\n\nGrok 4.7 asked as the person both times. Its worse build (0.1815) then sent this back to the sprint view:\n\n```\nreturn { sprint, ...totalsView(totals), hiddenCount: live.length - rows.length, changes: rows, members };\n```\n\n`members` is the sprint's full issue list, each row with its `issueKey`. In the recorded response BILL-306 is missing from `changes` and sitting at `members[25]`. Grok's better build doesn't return `members`, and passed.\n\nOpus, Sol and Jev Router kept v1's call as the person and send no member list, and all three passed the leak check with 18 of 18 hidden counts right in the builds I read. Each guards its admin page on the server by asking Jira for the caller's ADMINISTER permission. Opus answers \"Only Jira administrators (ADMINISTER) can use this page.\"\n\nGemini 4 Argon scored 0.7926 on its first Forge 2.0 build, the second best build on the board, and 0.2943 on its second. Both ran on the same goose build with the same settings and seeds. I traced what changed.\n\nAn upgrade has to copy v1's ledger rows into the new store (59 on the site I traced) and keep what v1 recorded about each one. In build 2 the admin panel's progress counter builds the migration list itself:\n\n``` js\nexport async function ensureMigrationCatalog(cfg) {\n  let mig = await kvs.get(MIGRATION_KEY);\n  if (mig && Array.isArray(mig.v1Keys)) return mig;\n  ...\n  mig = {\n    total: v1Keys.length,\n    migrated: 0,\n    complete: v1Keys.length === 0,\n    v1Keys,\n  };\n  await kvs.set(MIGRATION_KEY, mig);\n```\n\nOn the test site whose trace we kept, the first thing anyone did was open the admin page. The app's config didn't exist yet, so this wrote `{\"total\": 0, \"migrated\": 0, \"complete\": true}`, and nothing wrote that key again. The migration returned early for the rest of the run, and the hourly reconcile rebuilt the same changes from Jira's history and labelled them \"reconcile\". So 53 of the 59 rows had the wrong source label at each of the seven checkpoints, 53 times 7, the 371 corrupted rows the scorer reported. The rest of each row was right. The build's own comment, at line 710 of the same file, promises \"preserving v1's source\".\n\nBuild 1 has the same \"an empty list means done\" rule. But its panel only reads: it returns \"Migrated 0 of 0 v1 rows\" without writing anything, so the first list came from the scheduled sync, with all 59 rows. Build 2 did test its migration. It wrote a script that compared each row's source with v1's, and its reasoning noted \"achieving 88/88\" (its dev site had 88 rows). It always ran the sync before opening the admin page, so the scorer's order never happened there. My reading is that it comes down to which of two designs the model happened to write first. It took a second build and the scorer's order to expose the bad one.\n\nThe best answer to this moment came from the one model we ran on a laptop. The local Qwen3.8 27B base repairs exactly this case:\n\n```\n// Migration adoption: the v2 app first recorded the change (an event or reconcile that arrived before\n// the migration reached it), but the row's kind, issue, author and source must stay as v1 recorded\n// them. estimate/estimateField/boardId/deleted/deployedEnvs are kept from the v2 write (v1 had none).\nexport async function adoptV1Row(key, v1Row, enriched) {\n```\n\nand then overwrites `source: v1Row.source` on a row that got there first. Opus prevents it instead: its one row writer, used by every path, looks up v1's record before it creates a row (\"a change v1 recorded keeps v1's record\"). Both kept 59 of 59 rows intact. GLM-5.3 Prime's worse build broke the same field another way. Its hourly job reconciles before it migrates, so for the first two scored hours 82 rows on one of the three sites read \"reconcile\". Its other build migrates first. It has no critical. The same critical build answered all five sprint actions exactly on all three sites, which neither Sol build did on any.\n\nOur app takes a signed webhook from CI saying an issue was deployed. An unsigned or tampered call must get a 401 and change nothing, and one event sent twice at once must apply once. Three builds answered 401 correctly and still wrote something on each bad call, and in each of them the write sits in a wrapper around a signature check that works. In the local 27B base's second build, `dose` is the app's hourly budget of Jira calls, and `persist` saves it, signed request or not:\n\n``` js\nexport async function ciWebTrigger(event) {\n  const dose = await makeDose();\n  try {\n    const out = await handleCi(event, dose); // handleCi wraps the dose into a dosed Jira policy\n    await persist(dose);\n    return out;\n```\n\nDeepSeek V4.1 Flash, the September release that sits 14th, saves its counter the same way in a `finally` block, and GLM-5.3 FlashX writes its counter before it has even read the secret. DeepSeek's 17 October update made no write on a bad call in either build. On 13 October I [wrote](https://leanzero.net/blog/deepseek-v4-1-flash-coding-benchmark?utm_source=devto&utm_medium=referral&utm_campaign=crosspost) that the September build's critical was wrong, or at least far too expensive, since it's only a counter write after a correct 401. The scorer kept it, because the rule counts any write, and Opus, Sol and Jev show further down that the write isn't needed.\n\nThe same local build also checked whether it had seen an event, applied it, and only then wrote it down. Two copies arriving together both passed the check and both applied. GLM FlashX made the same choice and said so in a comment: \"Effect first (idempotent: environments are a set), then the once-only gate.\"\n\nThe local base's other build got that order right and lost the deployments anyway. It claimed each event first, Jira answered its next read with a 429, and the error was swallowed while the webhook still answered \"accepted\". A retry from CI then counts as a duplicate, and the deployment is gone for good, on two of the three test sites. DeepSeek's build, counter writes and all, does the step missing here. When the effect fails it deletes its claim: \"Never leave a half-applied event reserved: a retry must be able to record it.\"\n\nOpus, Sol and Jev got it all right. They read the secret first and compare the signature in constant time before writing anything. Then they claim the event with a Forge storage write that fails if the key already exists (`FAIL_IF_EXISTS`), so only one copy gets in. Opus deletes its claim if queuing fails. Sol and Jev store the event as `applied: false`, a to-do item that a worker flips once the deployment is recorded. Jev's build says why:\n\n```\n// The acceptance record IS the durable outbox; replays can safely repair a lost enqueue, never reapply a deployment.\ntry { await kvs.set(key, { body, applied: false }, { keyPolicy: 'FAIL_IF_EXISTS' }); }\n```\n\nSol's build writes the same `applied:false` record with nearly the same code. Whether Jev's routing beat Sol's own pair, or the newer goose build and luck did, two builds can't say. I wouldn't put a router in front of an agent because of it.\n\nThe two Prime models billed $36 to $52 a Forge build, MiMo V2.6 Flash $0.31. I expected the expensive ones to be doing more of something, and they weren't:\n\n| build | calls | bill | test or run commands | context per call | tests earned | score | \n|---|---|---|---|---|---|---|\n| GLM-5.3 Prime, better build | 295 | $47.76 | 99 | 272,000 | 0.895 | 0.4557 | \n| Qwen3.8 Max Prime, better build | 191 | $35.98 | 66 | 327,000 | 0.967 | 0.6515 | \n| MiMo V2.6 Flash, better build | 188 | $0.31 | 90 | 308,000 | 0.881 | 0.5538 | \n| Claude Opus 5.5, better build | 133 | $20.71 | 53 | 177,000 | 0.963 | 0.7294 | \n| GPT-6.1 Sol, better build | 47 | $1.15 | 27 | 87,000 | 0.950 | 0.6495 | \n\nThe two Primes sit in another comparison group. Treat the score gaps here as my reading. The Primes and MiMo, priced so differently, did the same kind of work at the same scale: 190 to 300 calls, about 300,000 tokens of context on each, dozens of test runs. They weren't re-reading files either, at most eight repeat reads per build. Both Primes cached over 99% of the prompt. A cached token cost $0.50 per million on Max Prime, $0.56 on GLM Prime and $0.0028 on MiMo. Cache reads were 86% of Max Prime's bill and 93% to 94% of GLM Prime's, and output was under $3 of either.\n\nDid the money buy score? Not much. Max Prime's better build earned 0.967 of the tests, the most of the five, and still finished below Opus on reliability, while GLM Prime earned less of the tests than Sol, who made 47 calls for $1.15. Near the end Max Prime caught a bug in its own checker (\"My check script had a member-collision bug (multiple members per issue). Re-validating properly:\"), and MiMo did the same in its second build (\"RACE: widget still loading! Need to wait for metric OR needs-config. My check was too eager.\"), for about a hundredth of the bill.\n\nDivide the average bill per build by the median and each 0.1 of Forge 2.0 score costs $0.06 with MiMo, $0.07 with DeepSeek's 17 October update (in another group), $0.20 with Sol, $3.15 with Opus and $11.58 with GLM-5.3 Prime. That update arrived while I was writing: 0.6957 on Gauntlet, 12th, and Forge 2.0 builds of 0.6618 and 0.2422 for about $0.31 each. I haven't read those builds yet. The chart has the 29 models with a bill for both builds. Gemini 4 Argon ran on Google's own API, and the local 27B has no API bill.\n\nEven Opus missed a paged read. Its first build checks a board's active sprints, to notice when one closes, with one request:\n\n``` js\nconst page = await jira(route`/rest/agile/1.0/board/${bid}/sprint?state=active&maxResults=50`, {}, {\n```\n\nThere's no loop. The test sites serve small pages, so the check missed sprints, though the app pages that endpoint properly elsewhere. On its worst site this build walked 50 of 108 multi-page reads to the end. No other build scoring 0.5 or more did worse. Jev Router's best build sends all its board and sprint lists through one helper that stops only when Jira says `isLast`, and walked 83 of 83. My guess is that two hand-written loops are where a long build drifts, and one shared helper would have stopped it.\n\nMost of the score gap on Forge 2.0 is in a reliability factor, and paging is only one check in it. A build's score is the tests it earned times that factor. The factor drops with the worst test in each group, on the worst of the three sites. The 16 builds that scored 0.5 or more earned a median 94% of the tests and kept a factor of 0.64, while the 24 builds between 0.2 and 0.5 earned 83% and kept 0.44, and five of them also took a critical defect. None of the top 16 did.\n\nMost of what you'd want to blame a behaviour on isn't in the vendors' cards.\n\n| model | architecture, per its own card | our \"medium\" against the vendor default | price per million, in / out / cached | \n|---|---|---|---|\n| GPT-6.1 Sol, Sol Pro | not published | the default | $2 / $10 / $0.10 | \n| Claude Opus 5.5 | not published | the default | $4 / $20 / $0.20 | \n| Claude Sonnet 5.5 | not published | one below (high) | $2 / $10 / $0.10 | \n| Claude Haiku 5.5 | not published | the default; Anthropic suggests high for longer agent tasks | $0.10 / $0.50 / $0.01, five times past 100k | \n| Gemini 4 Argon | sparse mixture of experts, no counts | the default | $2 / $10 until 30 November | \n| Grok 4.7 | \"a new, larger base model\" | one below (high) | $2 / $6 / $0.50 | \n| MiMo V2.6 Flash | 309B, 15B active; 39 of 48 layers see a 128-token window | no effort levels | $0.14 / $0.28 / $0.0028 | \n| Qwen3.8 Max Prime | fast tier of Qwen3.8 Max; open sibling 2.4T, 95B active | not published; the open sibling defaults to xhigh | $4 / $12 / $0.50 | \n| GLM-5.3 Prime | 256 experts plus 1 shared; total not stated | not a level (low, high, max) | $2.80 / $8.80 / $0.56 | \n| Qwen3.8 27B, our local base | dense 27B; 48 of 64 layers linear attention | one below (xhigh; there's no high) | local, 8-bit | \n| Jev Router | router, pool not published | picks its own effort | the routed model's | \n\nSources, all read on 18 October: [OpenAI's Sol page](https://developers.openai.com/api/docs/models/gpt-6.1-sol.md), [Anthropic's effort docs](https://platform.claude.com/docs/en/build-with-claude/effort), [Google's Argon card](https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-4-Argon-Model-Card.pdf), [xAI's Grok 4.7 post](https://x.ai/news/grok-4-7) and [reasoning docs](https://docs.x.ai/developers/model-capabilities/text/reasoning), the Hugging Face cards for [MiMo](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL), [Max's open sibling](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B), [GLM-5.3](https://huggingface.co/zai-org/GLM-5.3) and [Qwen3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B), [Alibaba on Prime](https://www.alibabacloud.com/help/en/model-studio/prime-mode) and [OpenRouter on Jev](https://openrouter.ai/typesafe/jev-router). OpenRouter lists the closed models' quantisation as \"unknown\".\n\nThe Prime bills are a pricing story more than an architecture one. Alibaba's Prime mode is a fast tier, \"1.5 to 2 times\" the tokens per second of the standard API. It's billed per token like the standard API, at about twice the price in Alibaba's own table, and OpenRouter lists Max Prime at exactly twice Qwen3.8 Max. MiMo's card puts 39 of its 48 layers on a 128-token window, which should make a long cached context cheap to hold. That may be why hosts can sell its cache reads 180 to 200 times cheaper. The card doesn't say so.\n\nGLM-5.3's open card says it runs any effort other than low or high as max, and Alibaba hasn't published how its Prime maps our \"medium\". Output was under $2 of each GLM Prime bill, so the reasoning tokens weren't what cost money. Whether a higher effort made it take more calls, one setting can't tell us.\n\nGoogle says [Gemini 4 Argon ignores temperature and top_p](https://ai.google.dev/gemini-api/docs/latest-model), so we couldn't have pinned sampling between its two builds. I doubt it would have made two 300-call builds alike anyway. Sol's builds billed about 14,000 reasoning tokens each, Opus's first 595,000, and OpenAI publishes nothing about Sol's output length.\n\nOur local base is the plain 8-bit Qwen3.8 27B we [fine-tuned for Forge](https://leanzero.net/blog/first-atlassian-trained-model-qwen-atlassian?utm_source=devto&utm_medium=referral&utm_campaign=crosspost) in September, published as `Mihai-LeanZero/Qwen3.8-27B-Atlassian-Q8-base-mlx`. It ran on our own MLX engine through a development build of the goose app. The board shows its pair as [one local entry](https://leanzero.net/agentic-benchmarks/forge/results/brun-4616377e-8e7f-4f34-9ca3-26c2dd8b7d46?utm_source=devto&utm_medium=referral&utm_campaign=crosspost).\n\nOn 12 October [I wrote](https://leanzero.net/blog/claude-haiku-5-5-vs-gpt-6-luna?utm_source=devto&utm_medium=referral&utm_campaign=crosspost) that if I were picking a cheap model to build something by itself, I'd pick Haiku 5.5. I even quoted [Anthropic's line](https://www.anthropic.com/claude-haiku-5-5) in that piece. Sonnet 5.5 and Opus 5.5 \"remain better choices for complex agentic coding tasks\", and Haiku is \"best suited to more narrowly scoped tasks\". I gave the pick anyway. For Forge 1.0-sized apps I'd still say it. On Forge 2.0 it's 0.0621.\n\nOn 8 October I called [Qwen3.8 Flash](https://leanzero.net/blog/qwen3-8-flash-vs-max-prime-forge-benchmark?utm_source=devto&utm_medium=referral&utm_campaign=crosspost) the Forge pick on a budget, with the docs in its context. Forge 2.0 gives the model no docs. Flash billed at least $1.68 a build for a 0.4077 median and MiMo $0.31 for 0.5283, in a different comparison group, so on this board my budget pick is MiMo.\n\nOur own mistakes are a longer list.\n\nOur scorer crashed on a call-heavy app, and the crash hid penalties. On 18 October the Forge probe threw a stack overflow on one test site for the local 27B's second build. 39 checks went unavailable, and the build showed an unpublishable 0.2736. After the fix the re-score is 0.1128, because the blanked checks were the webhook ones, and they held two criticals.\n\nOur secret probe is still wrong. On a build from the earlier local pair it took the admin's account ID for the CI secret and charged a critical. Neither build on the board now tripped it, and it stays wrong until a scorer release fixes it.\n\nThe board kept moving under this article. When the first local pair, 0.1457 and 0.1342, was retracted on 18 October, our research sheet carried its webhook line (\"valid: status 401\") over to the new pair. So I first had the 0.2576 build down as rejecting validly signed webhooks. Its code had accepted them and lost them to the swallowed 429. Then DeepSeek's update landed on Forge 2.0 while I was writing, and on Gauntlet half an hour later. Six more models arrived before it went out, and Pareto 26.10 pushed MiMo from 7th to 8th on Gauntlet.\n\n\"Medium\" isn't one setting. We pin each Forge 2.0 build at medium effort to keep the setup fixed. For Sol, Opus, Haiku and both Geminis that's the vendor default. Anthropic's effort docs say to start Haiku at medium for agentic coding and to use high for longer agent tasks, and ours is both. Sonnet and Grok ran a level below their defaults, and Qwen3.8 27B one below on a scale that skips high. GLM-5.3 and the Kimi K3 under Ember 1 don't have a medium, and MiMo has no effort parameter on OpenRouter at all. The Qwen card warns that lower effort \"can also lead to insufficient analysis, more failures, and repeated retries\".\n\nAnd goose ends a headless build on a turn with no tool call, while Anthropic documents that its models sometimes end a turn with a progress update. Haiku's builds died at that point, and so did one build of the cloud Qwen3.8 27B, after 18 calls (\"Let me look at the ModuleSchema definition\"). A Stop hook, a nudge or Anthropic's own prompt text might have changed Haiku's score.\n\nLiquid's LFM2.5 24B-A2B scored 0.0187 on Gauntlet in one run on our own engine, after a plain-text tool call our new parser didn't convert, so that's no verdict on LFM yet.\n\n[Opus and Sol for Forge](https://leanzero.net/blog/claude-sonnet-5-5-vs-opus-5-5?utm_source=devto&utm_medium=referral&utm_campaign=crosspost), my 5 October call, held up. MiMo before DeepSeek on a budget, from 13 October, is open again: the update costs the same $0.31 a build, and in another group its median sits 0.08 below MiMo's. I haven't read its builds.\n\nFrom Monday 19 October we run a weekly new-model review through both benchmarks, locally when a model fits on our Macs and through OpenRouter when it doesn't. I most want a Forge 2.0 pair for GPT-6.1 Sol Pro and the fine-tuned Atlassian 27B against its own base. After that, a Jev Router rematch beside a fresh Sol pair on one goose build.\n\nEach run page has the video and screenshots of its build, so check what was built before trusting a score here. You may read the same code and blame our harness instead, and on Haiku you might be right. Our goose fork isn't production ready (we use it because we know what's in it), so every run page is public and you can check us. Do you leave long builds running with nobody watching, in goose, Claude Code or your own agent? Has one ever told you it would do the rest next and then stopped? Tell us on our [Discord](https://discord.gg/RvYbd9qEUT) or at [office@leanzero.net](mailto:office@leanzero.net), and tell us too if a number here is wrong.\n\nOriginally published on [leanzero.net](https://leanzero.net/blog/best-model-for-agentic-coding-october-2026?utm_source=devto&utm_medium=referral&utm_campaign=crosspost). More Atlassian, Forge and local-AI write-ups at [leanzero.net/blog](https://leanzero.net/blog?utm_source=devto&utm_medium=referral&utm_campaign=crosspost), and if you're planning a migration or a Forge app, that's what we do: [leanzero.net/services](https://leanzero.net/services?utm_source=devto&utm_medium=referral&utm_campaign=crosspost).", "url": "https://wpnews.pro/news/best-model-for-agentic-coding-october-2026-benchmarks", "canonical_source": "https://dev.to/mihai_leanzero/best-model-for-agentic-coding-october-2026-benchmarks-3l2i", "published_at": "2026-10-20 05:12:01+00:00", "updated_at": "2026-10-20 05:17:17.578323+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-tools"], "entities": ["GPT-6.1 Sol", "Claude Opus 5.5", "Qwen3.8 Max Prime", "MiMo V2.6 Flash", "goose", "Jira", "Atlassian", "DeepSeek"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/best-model-for-agentic-coding-october-2026-benchmarks", "markdown": "https://wpnews.pro/news/best-model-for-agentic-coding-october-2026-benchmarks.md", "text": "https://wpnews.pro/news/best-model-for-agentic-coding-october-2026-benchmarks.txt", "jsonld": "https://wpnews.pro/news/best-model-for-agentic-coding-october-2026-benchmarks.jsonld"}}