Head to head: Sakana: Fugu Ultra vs GLM 5.2

Sakana: Fugu Ultra beat GLM 5.2 by a score of 107.9 to 96.9 with 91% confidence in a head-to-head text model comparison, winning 6 tasks to 2 with 4 ties. The evaluation used 12 fresh text tasks judged twice by OpenAI: GPT-5.6 Sol to cancel position bias, and Fugu Ultra proved more robust and dependable across code, technical reasoning, and summarization tasks, while GLM 5.2 showed strengths in localization and constraint scheduling.

The topline is decisive: Fugu Ultra wins 107.9 to 96.9, with 91% confidence . That matches the task count too — 6 wins to 2, with 4 ties — and the pattern matters more than the raw margin. Fugu Ultra didn’t just edge ahead on style points; it repeatedly produced answers that were more robust, more faithful to the prompt, and less likely to break in real use. The clearest separation showed up in code and technical reasoning. Fugu Ultra was better on the Python cache bug, the JavaScript group-by-day task, and the dense-passage summary because it handled edge cases and underlying mechanics more cleanly. It also took meeting-notes summarization and precise proofreading by being more complete and more exact. That’s the profile of a model that tends to do the unglamorous but important work correctly: sorting logic, UTC handling, faithful compression, and format-sensitive edits. GLM 5.2 did earn real wins, but they were narrower. It was better on the localization task, where its European Spanish toast copy sounded more natural and mobile-friendly, and on the constraint-scheduling puzzle, where it explained the uniqueness of the solution more convincingly. Those are legitimate strengths — fluency and tidy justification — but they weren’t enough to offset Fugu Ultra’s broader advantage on higher-stakes execution. The ties don’t rescue GLM. On LRU cache and warehouse reasoning, the judging notes themselves show some inconsistency across passes, which is exactly why they ended as ties rather than meaningful points for either side. On contradiction-finding and SQL, both models were simply good enough. Strip those out, and the remaining signal still points one way: Fugu Ultra was the stronger, more dependable model across this test set. Final call: Sakana: Fugu Ultra is the better text model here, and this is a clear win — not a squeaker. GLM 5.2 has nicer touch in a couple of language-heavy moments, but Fugu Ultra is the one I’d trust more across mixed real-world workloads. How they were tested We ran 12 fresh text tasks, generated on the fly for this matchup so neither model could prepare in advance, and had OpenAI: GPT-5.6 Sol score each one. To cancel position bias, every task was judged twice — once in each presentation order — and every number reported here, including the headline totals, is the average of both passes. Sakana: Fugu Ultra scored 107.9 to GLM 5.2's 96.9. 1. Localization with tone Translate this app onboarding line into natural, friendly European Spanish suitable for a mobile toast keep it under 60 characters, no exclamation marks : "You're all set — your first backup starts tonight." Return only the translation, then the character count in parentheses. Winner: GLM 5.2 — Model B sounds more natural and friendly for a mobile toast in European Spanish while staying concise and within the limit. Model A is accurate, but "copia de seguridad" feels heavier and less toast-like than Model B’s smoother phrasing. Second judge pass, order swapped — scores are the average of both: Model B is more natural and toast-friendly in European Spanish, with a concise, friendly tone and an accurate character count under the limit. Model A is also correct and within length, but "Ya está" sounds slightly less idiomatic here and "copia de seguridad" is bulkier for a mobile toast. 2. python-cache-bug Debugging & code review Python : The function should return the top k product IDs by total quantity sold, sorted by quantity descending then product ID ascending. It sometimes returns duplicates or misses products. Find the bug, fix it, and return the corrected code only. python from collections import defaultdict def top k products lines, k : totals = defaultdict int for product id, qty in lines: totals product id += qty ranked = for product id, total in totals.items : ranked.append total, product id ranked.sort reverse=True if len ranked = k: ranked = ranked :k-1 return product id for total, product id in ranked Example: lines = "P7", 2 , "P2", 5 , "P7", 4 , "P3", 5 , "P2", 1 expected top k products lines, 2 == "P2", "P7" Winner: Sakana: Fugu Ultra — Model A fully fixes the bug and returns the correct ordering by total descending then product ID ascending, while also handling nonpositive k safely. Model B fixes the truncation bug but keeps an incremental sort approach and does not handle k<=0 explicitly; it is correct for typical cases but less robust and less clean. Second judge pass, order swapped — scores are the average of both: Model A fully fixes the bug with a simpler and clearly correct approach: it aggregates totals, sorts by quantity descending then product ID ascending, and returns the top k IDs. Model B fixes the off-by-one truncation and tie-breaking, but it keeps unnecessary incremental sorting logic and does not handle nonpositive k explicitly, so Model A is the stronger answer. 3. js-group-by-day Practical coding JavaScript : Write a function groupErrorsByDay events that takes an array of objects like { ts: "2025-03-14T23:58:02Z", service: "billing", level: "error" } and returns an object mapping each UTC date YYYY-MM-DD to the count of events whose level is exactly "error" . Ignore non-error events. Example output: { "2025-03-14": 3, "2025-03-15": 1 } . Return code only. Winner: Sakana: Fugu Ultra — Model A is better because it correctly groups by UTC date using Date parsing and toISOString , and it safely skips invalid timestamps while counting only exact "error" levels. Model B is simpler but can misgroup events when the timestamp includes a timezone offset instead of already being in UTC, and it also violates the "code only" instruction by wrapping the answer in Markdown fences. Second judge pass, order swapped — scores are the average of both: Model A is more robust because it derives the UTC day via Date parsing and toISOString , which correctly handles timestamps with offsets as well as Z timestamps, while Model B assumes the first 10 characters always represent the desired UTC date. Both follow the instruction to return code only in their outputs, but Model A is the higher-quality solution overall. 4. meeting-notes-summary Summarization: Summarize the meeting notes below into EXACTLY 4 bullet points. Each bullet must start with "- " and be no more than 16 words. Be faithful; do not add anything. Source notes: Project: Northline clinic scheduling portal - Mara said the pilot at Cedar Hill starts on 12 May, not 5 May as last week's deck showed. - Engineering finished SMS reminders, but Spanish localization is still missing for 7 message templates. - Devin reported the patient import bug only affects files exported from MedAxis v3.2; workaround is to save as CSV UTF-8. - Support wants a one-page troubleshooting guide before launch because front-desk staff turnover is high. - Legal approved the consent text after removing the sentence about "marketing outreach". - Open risk: no confirmation yet from Cedar Hill on who will train weekend reception staff. - Next check-in is Tuesday 09:30. Winner: Sakana: Fugu Ultra — Model A better covers the source faithfully within exactly four bullets, including the legal approval, open training risk, and next check-in. Model B follows the format but omits several important notes and slightly weakens the workaround detail by dropping the UTF-8 specification. Second judge pass, order swapped — scores are the average of both: Model A better covers the source faithfully within exactly four bullets, including legal approval, the open training risk, and the next check-in. Model B follows the format but omits several important notes and weakens one detail by changing the workaround from CSV UTF-8 to just CSV. 5. LRU cache Implement a class LRUCache<K, V in TypeScript with a fixed capacity set in the constructor, and O 1 get key : V | undefined and set key, value : void . Accessing or updating a key must mark it most-recently-used; inserting beyond capacity must evict the least-recently-used entry. Return ONLY the class, no prose. Winner: Tie — Model A better follows the prompt by returning only the class and implements the required LRU behavior with O 1 operations using Map insertion order. Model B is also largely correct, but it violates the output-format instruction by including Markdown code fences and adds an unsolicited constructor constraint that changes behavior for nonpositive capacities. Second judge pass, order swapped — scores are the average of both: Model B fully satisfies the prompt with a valid fixed-capacity LRU cache and returns only the class. Model A is mostly correct, but it does not enforce a valid fixed capacity in the constructor and instead silently no-ops when capacity is nonpositive, making its behavior less robust for the stated requirement. 6. Constraint scheduling Four talks A, B, C, D fill four 1-hour slots 9,10,11,12. Constraints: A is before D; C is not first; B is immediately after A; D is not at 12. Give the ONE valid schedule as 'slot: talk' lines, then a one-line justification. If impossible, say so and explain. Winner: GLM 5.2 — Model B is better because it gives the same correct unique schedule and also briefly explains why it is the only valid arrangement. Model A is correct and follows the format, but its justification only verifies the final schedule rather than establishing uniqueness as clearly. Second judge pass, order swapped — scores are the average of both: Model B and Model A give the same valid schedule, but Model B is better because its justification explains why the schedule is uniquely forced, which more fully satisfies the prompt. Model A only verifies the listed schedule without establishing uniqueness. 7. Summarize dense passage Summarize the passage below in exactly three bullet points, each one sentence, capturing the mechanism, the tradeoff, and the caveat — no jargon a non-specialist couldn't follow. Return only the bullets. Passage: "Speculative decoding pairs a small draft model with a large target model: the draft proposes several tokens, the target verifies them in one pass, and accepted tokens are kept while the first rejection resets to the target's own choice. This can cut latency substantially when acceptance is high, but the draft model's compute is pure overhead when acceptance is low, and gains evaporate on adversarial or highly novel inputs where the draft and target disagree often." Winner: Sakana: Fugu Ultra — Model A more precisely captures the mechanism, including that only the first rejected token is replaced by the larger model’s own choice, and it cleanly covers the tradeoff and caveat in plain language. Model B is clear and readable, but it omits that reset detail and overstates the caveat by saying benefits "completely disappear" and disagreements happen "constantly," which is stronger than the passage. Second judge pass, order swapped — scores are the average of both: Model A is better because it more precisely preserves the passage’s mechanism, including that approved pieces are kept and the first rejected one is replaced by the larger model’s own choice, while still staying clear for non-specialists. Model B is strong and readable, but it is slightly less exact about the rejection/reset behavior and adds a more absolute claim with "completely disappear." 8. Precise proofreading Correct ONLY the grammar, spelling, and punctuation errors in the sentence below without changing wording, tone, or meaning, then on a second line list each change as 'was - now'. Sentence: "Its been a long time since we seen there new office, and me and him was suprised how quite it was." Winner: Sakana: Fugu Ultra — Model A correctly fixes the sentence and mostly follows the requested change-list format, though it uses semicolons instead of listing each change exactly as 'was - now'. Model B fixes the sentence itself, but its second line is incorrect because every listed change starts with 'was' rather than the original text, so it fails the core formatting/content requirement. Second judge pass, order swapped — scores are the average of both: Model A is better because it makes the same necessary corrections as Model B while formatting the change list more clearly and accurately as explicit original-to-corrected pairs. Model B’s second line does not follow the requested 'was - now' format meaningfully, since it repeats 'was - ' for every item instead of listing each actual change. 9. Find the contradiction The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones." Winner: Tie — Model A and Model B both identify the same two conflicting sentences verbatim and explain the contradiction accurately in one sentence. Model B is slightly more formatted, but both fully satisfy the task with equally strong correctness and clarity. Second judge pass, order swapped — scores are the average of both: Model B and Model A both identify the same two conflicting sentences verbatim and explain the conflict accurately in one sentence without attempting to fix it. Model B is slightly more explicitly structured, while Model A is slightly more concise, but neither is meaningfully better overall. 10. customer-delay-email Professional writing: Draft an email to a customer named Priya Nair at BrightSketch. Audience: an operations lead who is frustrated. Context: her team's custom invoice export was promised for Friday, but QA found a rounding issue in tax totals affecting some EU invoices. New ETA is next Wednesday. We can enable a temporary CSV export today that includes all needed fields but requires manual filtering. Tone: candid, accountable, calm. Length: 120-150 words. Include a clear apology, the revised ETA, the workaround, and an offer for a 15-minute call. Winner: Sakana: Fugu Ultra — Model A is slightly better because it is more candid and accountable, explicitly states responsibility, gives the revised ETA and workaround clearly, and offers the 15-minute call in a calm, professional tone. Model B is strong, but "I take full responsibility" feels less precise in a team context and it is a bit less concrete about next steps than Model A. Second judge pass, order swapped — scores are the average of both: Model A is slightly better because it is more direct and accountable, explicitly ties the impact to Priya’s team, and presents the revised ETA and workaround with strong clarity while maintaining a calm professional tone. Model B is also strong, but its phrasing is a bit more generic and slightly less crisp for a frustrated operations lead. 11. subscription-sql SQL & data queries: Using the schema below, write a single SQL query to return each account manager's name and the total MRR from accounts that were active on 2024-11-30. An account is active on that date if start date <= '2024-11-30' and end date is null or end date '2024-11-30' . Only include managers whose active-account MRR total is at least 5000. Return columns: manager name , active mrr . Sort by active mrr descending, then manager name ascending. Schema: - account managers id, manager name - accounts id, manager id, start date, end date, monthly recurring revenue Winner: Tie — Model A and Model B both produce a correct single-query solution that applies the active-account date logic, aggregates MRR by manager, filters totals of at least 5000, returns the requested columns, and sorts correctly. Model A is slightly more explicit with DATE literals, but Model B is equally valid for the task as written. Second judge pass, order swapped — scores are the average of both: Model B and Model A both produce a correct single-query solution that applies the active-account date logic, aggregates MRR by manager, filters totals with HAVING, returns the requested columns, and sorts as instructed. Model A is slightly more explicit with DATE literals, but that does not make it materially better for this task. 12. warehouse-reasoning Step-by-step reasoning: A small warehouse packs three orders using rolls of tape from one shared shelf. - Shelf starts with 9 full rolls and 1 partial roll with 18 meters left. - Order A uses 2 full rolls plus 11 meters from another roll. - Order B uses 37 meters total. - Order C uses 3 full rolls plus 24 meters from another roll. - Every full roll contains 30 meters. - Workers always finish the current partial roll before opening a new roll. After all three orders, how many meters of tape remain on the shelf? Give just the final number. Winner: Tie — Model A is correct: the shelf starts with 288 meters total, and following the rule about using the current partial roll first leaves 66 meters after Orders A, B, and C. Model B gives an incorrect final number, though both models do follow the instruction to provide just the final number. Second judge pass, order swapped — scores are the average of both: Model B is correct: starting with 288 meters total, applying the partial-roll rule gives remaining tape of 23 meters after Orders B, A, and C. Model A's 66 is incorrect, though both models did follow the instruction to give just the final number. See every prompt and the full side-by-side outputs in the interactive Head-to-Head /head-to-head/head-to-head-sakana-fugu-ultra-vs-glm-5-2 .