{"slug": "ornith-1-5-122b-benchmarks-how-it-stacks-up-on-coding-and-agentic-tests", "title": "Ornith-1.5 122B Benchmarks: How It Stacks Up on Coding and Agentic Tests", "summary": "Ornith-1.5 122B, an MIT-licensed open-weight mixture-of-experts model with roughly 122 billion total parameters and a 260,000-token context window, leads its own benchmark tables on coding tasks, scoring 76.9 on SWE-bench Pro (no-hack) against 65.5 for Qwen3.8 dense, 61.0 for Nemotron 3.5 Super VL, 43.9 for Qwen3.8-Flash-Next and 57.0 for the 501B-parameter Beam. The Ornith team's model card reports 84.5 on SWE-bench Verified and 76.4 on SWE-bench Multilingual, plus 78.5 on BrowseComp and 75.2 on WideSearch, where DeepSeek-V4-Flash edges it out narrowly. Ornith-1.5 122B trails DeepSeek-V4-Flash (284B) and Beam (501B) on the HLE reasoning benchmark, and independent hands-on testing broadly supports the results while finding low-resource language translation remains weak.", "body_md": "# Ornith-1.5 122B Benchmarks: How It Stacks Up on Coding and Agentic Tests\n\nOrnith-1.5 122B's own benchmark tables versus Qwen3.8, Nemotron 3.5 Super, DeepSeek-V4-Flash, and Beam on SWE-bench, TerminalBench, and more.\n\n## What is Ornith-1.5 122B?\n\nOrnith-1.5 122B is an open-weight mixture-of-experts language model released under the MIT license by the Ornith team. It has around 122 billion total parameters, reasons through a chain of thought before answering, handles both text and images, and supports a context window of roughly 260,000 tokens. It’s built on top of the earlier Ornith-1.0 lineage (itself developed from Qwen3.5 and Gemma4 with further pretraining and post-training), and the 1.5 generation adds a self-improvement loop that jointly optimizes task generation, agent scaffolding, and solution rollouts rather than relying only on fixed human-curated training tasks.\n\n## TL;DR\n\n- **Ornith-1.5 122B is MIT-licensed and open weight** , meaning anyone can download, run, and modify it without a commercial license barrier.\n- On **SWE-bench Pro (no-hack)** it scores 76.9, clearly ahead of Qwen3.8-Flash-Next (43.9), Qwen3.8 dense (65.5), Nemotron 3.5 Super VL (61.0), and even the 501B-parameter Beam model (57.0).\n- It also leads on **SWE-bench Verified** (84.5) and**SWE-bench Multilingual** (76.4), outperforming every listed rival on those two coding tasks.\n- On agentic benchmarks like **BrowseComp** (78.5) and**WideSearch** (75.2) it beats most same-size and larger competitors, though DeepSeek-V4-Flash edges it out narrowly on both.\n- On the **HLE (Humanity’s Last Exam) reasoning benchmark** , Ornith-1.5 122B trails DeepSeek-V4-Flash and Beam, both of which are far larger models (284B and 501B parameters respectively).\n- The model card frames the release around three themes: **stronger, faster, and leaner** , meaning better instruction-following, shorter outputs, and quicker task completion relative to Ornith-1.0.\n- Independent hands-on testing (coding fix, vision, multilingual translation) broadly supports the benchmark story, though low-resource language translation remains weak.\n\n## Remy is new. The platform isn't.\n\nRemy is the latest expression of years of platform work. Not a hastily wrapped LLM.\n\n## What benchmarks does the Ornith-1.5 122B model card report?\n\nThe model card groups its results into three categories: coding, reasoning, and agentic tasks. It compares Ornith-1.5 122B against five other models of varying sizes: Qwen3.8-Flash-Next (125B), Qwen3.8 dense (27B), Nvidia’s Nemotron 3.5 Super VL (120B), DeepSeek-V4-Flash-0731 (284B), and a much larger model called Beam (501B).\n\nIn coding, the tested benchmarks are TerminalBench 2.1, CyberGym, NL2Repo, and three flavors of SWE-bench (Pro no-hack, Verified non-hack, and Multilingual non-hack). In reasoning, the card reports HLE with and without tool use, plus SciCode. In agentic evaluation, it lists WideSearch, BrowseComp, MCP-Atlas, Toolathlon-Verified, and ClawEval.\n\nThis spread matters because it tests different failure modes: SWE-bench variants check whether a model can fix real GitHub issues in actual repositories, TerminalBench checks command-line task completion, HLE checks broad expert-level reasoning, and the agentic suite checks multi-step tool use and web research, which is a different skill from single-turn code generation.\n\n## How does Ornith-1.5 122B perform on coding benchmarks?\n\nThe strongest numbers in the whole model card are in coding. On SWE-bench Pro (no-hack), Ornith-1.5 122B posts 76.9, well clear of Qwen3.8-Flash-Next (43.9), Nemotron 3.5 Super VL (61.0), and Qwen3.8 dense (65.5). Beam, at roughly four times the parameter count, scores 57.0 on the same test, meaning the much smaller Ornith model outperforms it here.\n\nOn SWE-bench Verified, Ornith-1.5 122B scores 84.5, ahead of Qwen3.8-Flash-Next (82.8) and Qwen3.8 dense (76.8). Neither DeepSeek-V4-Flash nor Beam report a Verified score in the table. The pattern repeats on SWE-bench Multilingual, where Ornith leads at 76.4 against Qwen3.8-Flash-Next’s 73.0.\n\nTerminalBench 2.1 is the one coding test where Ornith-1.5 122B doesn’t lead. It scores 81.2, which is close to but behind Qwen3.8-Flash-Next (83.4) and DeepSeek-V4-Flash (82.7), and ahead of Beam (80.1). CyberGym and NL2Repo both go in Ornith’s favor against most rivals, though DeepSeek-V4-Flash edges ahead on NL2Repo (54.2 versus Ornith’s 51.2).\n\nThe video creator’s hands-on test lines up with these numbers. Running the model locally via llama.cpp against a buggy legacy application, Ornith-1.5 122B located and fixed a one-line date comparison bug in an inheritance calculation app without being told where to look, correctly flipping the payout logic for beneficiaries who died before versus after the relevant person.\n\n## How does it do on reasoning tests like HLE?\n\nReasoning is where the size gap starts to show. On HLE with tools, Ornith-1.5 122B scores 48.0, beating Qwen3.8-Flash-Next (38.0), Qwen3.8 dense (31.3), and Nemotron 3.5 Super VL (33.2), but trailing DeepSeek-V4-Flash (50.8) and Beam (51.1), both substantially larger models. The gap without tools is similar: Ornith scores 37.68, ahead of same-size competitors but without a direct comparison figure from the two largest models in that row.\n\nOn SciCode, a benchmark aimed at scientific coding and reasoning, Ornith-1.5 122B scores 53.8, ahead of every other model in the table, including Qwen3.8-Flash-Next (47.5) and Nemotron 3.5 Super VL (47.1).\n\nThe pattern suggests Ornith-1.5 122B is tuned more heavily toward applied reasoning tasks tied to code and tools than toward open-ended knowledge-heavy exam questions, where sheer parameter count still seems to help.\n\n### Everyone else built a construction worker.\n\nWe built the contractor.\n\nOne file at a time.\n\nUI, API, database, deploy.\n\n## How does it compare on agentic and tool-use benchmarks?\n\nThe agentic suite is where Ornith-1.5 122B’s “built for real work” pitch gets tested directly. On BrowseComp, a web-browsing and research benchmark, it scores 78.5, ahead of Qwen3.8-Flash-Next (75.3) and Nemotron 3.5 Super VL (61.0), though behind Beam’s 80.2. On WideSearch it scores 75.2, again behind Beam (77.3) but ahead of the rest.\n\nOn MCP-Atlas, which tests tool-calling through the Model Context Protocol, Ornith ties with Beam at 74.6 and edges out Qwen3.8-Flash-Next (74.1). On Toolathlon-Verified, it scores 70.8, just behind Qwen3.8-Flash-Next’s 71.4. ClawEval figures for most competitors are cut off in the available data, but Ornith posts 79.4 on that test.\n\nTaken together, Ornith-1.5 122B doesn’t win every agentic benchmark, but it consistently lands either ahead of or within a few points of models two to four times its size. That’s the core claim the model card is making: competitive agentic performance at a fraction of the parameter count, which matters directly for deployment cost and local hosting.\n\n## Is Ornith-1.5 122B worth running locally?\n\nFor teams that want an open-weight model they can self-host, the case is reasonably strong based on both the benchmark tables and independent testing. It’s MIT-licensed, runs through standard local inference stacks like llama.cpp, and is available in quantized GGUF-style formats, with VRAM requirements scaling depending on the chosen quant and context length. A smaller quantized variant, Ornith-1.5-122B-Mobile, is also mentioned as edge-deployable.\n\nIn practical testing, the model handled a multi-file debugging task autonomously, produced an accurate and detailed response to a food-identification vision prompt (correctly naming a North Indian chicken and potato curry dish and writing a full recipe), solved a basic circuit-reasoning puzzle from an image, and generated a complex single-file HTML animation without external dependencies. Multilingual translation across roughly 80 languages was accurate for major world languages but noticeably weaker for low-resource languages like Thai, Zulu, and regional South Asian languages, a common limitation across most current multilingual models.\n\n## Frequently Asked Questions\n\n### How many parameters does Ornith-1.5 122B have?\n\nIt has approximately 122 billion total parameters and uses a mixture-of-experts (MoE) architecture, meaning only a subset of parameters activates for any given token, which helps keep inference costs down relative to a dense model of the same size.\n\n### Is Ornith-1.5 122B open source?\n\nYes. It’s released under the MIT license as an open-weight model, so it can be downloaded, run, and modified without the licensing restrictions some other open models impose.\n\n### What is SWE-bench Pro and why does it matter?\n\nSWE-bench Pro is a coding benchmark built from real software issues in actual repositories, testing whether a model can understand a bug report and produce a correct code fix. The “no-hack” variant filters out solutions that pass tests through shortcuts rather than genuine fixes, making it a stricter measure of real-world coding capability.\n\n### Does Ornith-1.5 122B beat larger models like DeepSeek-V4-Flash and Beam?\n\nIt beats them on several coding and agentic benchmarks, including SWE-bench Pro and CyberGym, despite being smaller than both. It trails them on reasoning-heavy benchmarks like HLE, where the larger parameter counts of DeepSeek-V4-Flash (284B) and Beam (501B) appear to give an edge.\n\n### What are the main weaknesses of Ornith-1.5 122B?\n\nBased on available benchmarks and testing, it lags slightly behind top rivals on TerminalBench 2.1 and some tool-use tests like Toolathlon-Verified, and its multilingual output quality drops noticeably for low-resource languages outside major world languages.", "url": "https://wpnews.pro/news/ornith-1-5-122b-benchmarks-how-it-stacks-up-on-coding-and-agentic-tests", "canonical_source": "https://www.mindstudio.ai/blog/ornith-1-5-122b-benchmarks-coding-agentic/", "published_at": "2026-10-21 00:00:00+00:00", "updated_at": "2026-10-21 13:13:33.970308+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-products", "ai-agents"], "entities": ["Ornith-1.5 122B", "Ornith", "Qwen3.8", "Nemotron 3.5 Super VL", "DeepSeek-V4-Flash", "Beam", "Nvidia", "SWE-bench"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ornith-1-5-122b-benchmarks-how-it-stacks-up-on-coding-and-agentic-tests", "markdown": "https://wpnews.pro/news/ornith-1-5-122b-benchmarks-how-it-stacks-up-on-coding-and-agentic-tests.md", "text": "https://wpnews.pro/news/ornith-1-5-122b-benchmarks-how-it-stacks-up-on-coding-and-agentic-tests.txt", "jsonld": "https://wpnews.pro/news/ornith-1-5-122b-benchmarks-how-it-stacks-up-on-coding-and-agentic-tests.jsonld"}}