Head to head: DeepSeek-V4-Pro vs Llama-4-Maverick-17B-128E-Instruct-FP8 DeepSeek-V4-Pro defeated Llama-4-Maverick-17B-128E-Instruct-FP8 with an aggregate score of 101.3 to 86.2, winning 9 of 12 tasks with 98% confidence in a head-to-head benchmark conducted by an unnamed evaluator using gpt-5.4 as judge. DeepSeek excelled in correctness, compliance, and real-world reliability, while Llama showed formatting discipline but suffered self-inflicted errors like adding prose and producing broken SQL. DeepSeek-V4-Pro wins this head-to-head cleanly and convincingly. It posts a 101.3 to 86.2 aggregate advantage, takes 9 of 12 tasks , and does it with a 98% confidence verdict. That is not a marginal edge or a judge-quirk result; it’s a broad performance gap. What stands out is how DeepSeek won. It was repeatedly better at the unglamorous but essential stuff that decides real-world usefulness: returning only the SQL when asked, producing runnable SQL instead of broken window-query output, keeping localization natural and audience-appropriate, and writing tighter code in Python and TypeScript. On tasks like the SQL top unresolved bug component, Brazilian Portuguese outage localization, the window-function query, campaign-name normalization, unit-aware math, vendor-change extraction, and the LRU cache, DeepSeek was simply the more reliable model. Llama-4-Maverick-17B-128E-Instruct-FP8 did have two legitimate bright spots. It won strict JSON extraction by following the title-case normalization requirement more precisely, and it edged audit interview assignment —though that says more about DeepSeek’s self-contradictory failure than about Maverick producing a good answer, since both models ended with invalid schedules. There was also one tie on nuanced classification, a split-decision kind of result where both got the labels right but each stumbled on formatting in different ways. The bigger problem for Llama is that too many of its losses were self-inflicted. It added prose where the prompt explicitly asked for a single query, inserted unnecessary quotation marks in localization, and, most damagingly, produced a SQL window solution with a fatal column-reference error. Those are not stylistic nits; they are the kinds of misses that break downstream workflows. DeepSeek wasn’t perfect—there was at least one mixed judgment on the enterprise support rewrite and some disagreement on vendor-change extraction—but across the full slate it was the steadier, sharper model. Final call: DeepSeek-V4-Pro is the clear winner. Llama-4-Maverick-17B-128E-Instruct-FP8 shows flashes of strong formatting discipline, but DeepSeek was better where it matters most: correctness, compliance, and shipping answers that actually work. How they were tested We ran 12 fresh text tasks, generated on the fly for this matchup so neither model could prepare in advance, and had gpt-5.4 score each one. To cancel position bias, every task was judged twice — once in each presentation order — and every number reported here, including the headline totals, is the average of both passes. DeepSeek-V4-Pro scored 101.3 to Llama-4-Maverick-17B-128E-Instruct-FP8's 86.2. 1. Strict JSON extraction Extract every meeting from this text into a JSON array of {"title","day","start24","durationMinutes"}. Use 24-hour "HH:MM" for start24, integer minutes for duration, and title-case titles. Return ONLY valid JSON, no commentary. Text: "Standup is daily at 9am for a quarter hour. The design review runs thursday from 2:30-4pm. Payroll sync — first monday, 11:00 to noon." Winner: Llama-4-Maverick-17B-128E-Instruct-FP8 — Model B better satisfies the formatting instruction by using title case consistently for the textual fields, while Model A leaves the day values in lowercase. Both models correctly extract the meetings, times, and durations, but Model B adheres more closely to the requested normalization. Second judge pass, order swapped — scores are the average of both: Model B is better because it correctly normalizes the day values into title case while preserving the extracted schedule details. Model A has the same meeting times and durations, but it fails the title-case-style requirement for the day fields, making it less compliant with the formatting instructions. 2. SQL top unresolved bug component Write a single SQL query that answers this exactly: Return the component name and unresolved bug count for the component with the highest number of unresolved bugs created in Q1 2025 by non-internal reporters. If there is a tie, return the alphabetically earliest component name. Schema: components id INTEGER PRIMARY KEY, component name TEXT users id INTEGER PRIMARY KEY, email TEXT, is internal BOOLEAN issues id INTEGER PRIMARY KEY, component id INTEGER, reporter id INTEGER, kind TEXT, status TEXT, created at TIMESTAMP Rules: - "unresolved" means status NOT IN 'resolved','closed' - only kind = 'bug' - Q1 2025 means created at = '2025-01-01' AND created at < '2025-04-01' - non-internal reporters means users.is internal = FALSE Return one row only. Winner: DeepSeek-V4-Pro — Model A and Model B provide equivalently correct SQL logic, but Model A better follows the prompt by returning just a single SQL query with no extra prose. Model B adds explanatory text despite the instruction to write a single SQL query, so Model A is the better overall response. Second judge pass, order swapped — scores are the average of both: Model A provides a correct single SQL query that directly satisfies the prompt. Model B's SQL is also correct, but it violates the instruction to write a single SQL query by adding explanatory text and a heading, so Model A is better overall. 3. Localize outage notice to Brazilian Portuguese Translate this customer-facing notice into Brazilian Portuguese. Tone: calm, professional, empathetic. Length constraint: 55–70 words. Keep all specifics and the apology. "We found an issue that delayed some payout notifications between 14:10 and 15:35 UTC on 3 September. The payouts themselves were not affected, and no action is required from you. We have re-sent the missing notifications and are monitoring the queue closely. We're sorry for the confusion." Winner: DeepSeek-V4-Pro — Model A better preserves all specifics while sounding calm, professional, and empathetic; it also explicitly keeps the customer-directed detail that no action is required from the recipient. Model B is strong, but it drops the explicit "from you" nuance and adds unnecessary quotation marks, making it slightly less polished for the task. Second judge pass, order swapped — scores are the average of both: Model A is slightly better because it preserves more of the original meaning and customer-facing tone, especially with "de sua parte" for "from you" and "Os pagamentos em si" for "The payouts themselves." Model B is also strong, but it omits that nuance and its closing apology is a bit less direct than the source. 4. Rewrite enterprise support update Rewrite this internal note as a polished email to an enterprise customer. Audience: IT director at a university. Tone: accountable and reassuring, not defensive. Length: 120–150 words. Source note: "We rolled the SSO cert on the wrong app shard last night, so a subset of North Campus users couldn't log in from about 07:12 to 08:03 local time. Admin accounts and API tokens were fine. We reverted, validated with two faculty accounts, and queued the correct cert for tonight 23:30 with a rollback plan. No evidence of data access problems. Need to own the mistake, mention next update by 10:30, and avoid too much engineering jargon." Winner: DeepSeek-V4-Pro — Model A is more polished and reassuring while still owning the mistake, and it avoids excessive jargon better than Model B. Model B is solid but introduces a timing inconsistency “last night” vs. the stated local-time window and “by 10:30 today” , and its wording is slightly more technical and less tailored to the requested tone. Second judge pass, order swapped — scores are the average of both: Model B is better because it stays closer to the requested accountable, reassuring customer-facing tone and correctly includes the required next update timing. Model A is polished, but it introduces a likely timing error by saying the summary will arrive by 10:30 tomorrow instead of by 10:30, and its wording is slightly more technical than requested. 5. SQL window query Given Postgres tables orders id, customer id, created at, total cents and customers id, name , write ONE SQL query returning each customer's name and their SECOND-largest order total in dollars, 2 decimals , excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL. Winner: DeepSeek-V4-Pro — Model A correctly returns each customer's second-highest order using a window function, excludes customers with fewer than two orders implicitly, formats dollars to two decimals, and orders by that value descending. Model B has a correctness issue because the outer query references customer id even though it is not selected in ranked orders, so the SQL would fail. Second judge pass, order swapped — scores are the average of both: Model A correctly uses a window function to select each customer's second-highest order and naturally excludes customers with fewer than two orders. Model B has a fatal error because its outer query references customer id even though that column is not selected in the ranked orders CTE, so the SQL would not run as written. 6. Localization with tone Translate this app onboarding line into natural, friendly European Spanish suitable for a mobile toast keep it under 60 characters, no exclamation marks : "You're all set — your first backup starts tonight." Return only the translation, then the character count in parentheses. Winner: DeepSeek-V4-Pro — Model A is more natural and concise for a mobile toast, stays under 60 characters, and follows the requested output format without extra punctuation. Model B is understandable but less natural in European Spanish, includes quotation marks not requested, and uses a heavier phrasing for a toast message. Second judge pass, order swapped — scores are the average of both: Model A is more natural and toast-friendly in European Spanish, stays under 60 characters, and avoids unnecessary verbosity. Model B sounds less natural for onboarding, includes quotation marks despite the return-format instruction, and its stated character count appears incorrect while the text itself exceeds the limit. 7. Assign audit interviews Schedule four 30-minute audit interviews on Monday using these slots: 09:00, 09:30, 10:00, 10:30. One person per slot. Return the final schedule as JSON mapping slot to person. People and constraints: - Mara cannot do 09:00 and must be earlier than Tovin. - Tovin cannot do 10:30. - Elise must be either first or last. - Jae cannot do consecutive-to-Mara's slot adjacent times count as consecutive and cannot do 09:30. - The 10:00 slot must be filled by someone whose name contains the letter "e". There is exactly one valid schedule. Winner: Llama-4-Maverick-17B-128E-Instruct-FP8 — Model B is slightly better because, although it also ends with an invalid schedule, its reasoning is somewhat more coherent and less self-contradictory than Model A’s. Model A repeatedly proposes and retracts invalid schedules and finishes with a mapping that violates multiple constraints, making it less reliable overall. Second judge pass, order swapped — scores are the average of both: Model B and Model A both fail to produce a valid schedule and both violate the prompt’s requirement to return only a final JSON mapping. Model B is somewhat more coherent in its reasoning, while Model A is more self-contradictory, but neither is correct enough to win. 8. Python normalize campaign names Return code only. In Python, write a function normalize campaign name: str - str for analytics ingestion. Requirements: - Trim leading/trailing whitespace. - Convert to lowercase. - Replace any run of characters that is not a letter or digit with a single hyphen. - Remove leading/trailing hyphens. - If the result is empty, return 'untitled'. - Keep Unicode letters/digits e.g. café, 東京 rather than stripping them. - Include a few assert-based tests in the same code block. Do not use external packages. Winner: DeepSeek-V4-Pro — Model A fully satisfies the normalization requirements with a direct Unicode-safe implementation and includes several relevant assert-based tests. Model B is also correct, but its repeated string replacement approach is less efficient and slightly less polished for collapsing runs of non-alphanumeric characters. Second judge pass, order swapped — scores are the average of both: Model A better matches the requirement to replace each run of non-letter/digit characters with a single hyphen by doing it in one pass, while preserving Unicode alphanumerics correctly. Model B is mostly correct, but its repeated string replacement approach is less direct and efficient for collapsing runs, whereas Model A is cleaner and more precise overall. 9. Unit-aware math A pump moves 3.5 liters every 8 seconds. A tank holds 0.9 cubic meters. Starting empty, how long to fill it, in minutes and seconds mm:ss , rounded to the nearest second? Show the key steps, then give the final answer on its own line. Winner: DeepSeek-V4-Pro — Model A and Model B both compute the correct result, 34:17, and both show the key steps. Model A is slightly better because it is more concise and follows the formatting request more closely by presenting the final answer cleanly on its own line, whereas Model B adds extra prose around the final answer. Second judge pass, order swapped — scores are the average of both: Model A is slightly better because it is fully correct, concise, and follows the requested format more cleanly by showing the key steps and then giving the final answer clearly on its own line. Model B is also correct, but it is a bit more verbose and includes an extra lead-in before the final answer. 10. Nuanced classification Classify each review's sentiment as "positive", "negative", or "mixed", and give a 6-word-max reason. Return ONLY a JSON array of {"text","label","reason"} in input order. Reviews: "Fast shipping but the fabric feels cheap.", "Absolutely love it, wearing it daily ", "It broke after a week. Refund was quick and painless though." Winner: Tie — Model A and Model B both classify all three reviews correctly, but Model A follows the prompt more precisely by keeping every reason within the 6-word maximum. Model B exceeds that limit on the second reason, so Model A is better overall. Second judge pass, order swapped — scores are the average of both: Model B fully follows the required output format and correctly classifies all three reviews with concise reasons under six words. Model A is also correct on sentiment, but it violates the instruction to return only a JSON array by wrapping the answer in a Markdown code block. 11. Extract vendor change facts From the messy note below, extract the requested facts into JSON with keys: vendor name, legal entity, previous billing email, new billing email, contract id, effective date, annual cap usd, notice days, termination for convenience, support tier, sla uptime percent, security contact, dpa signed, tax residency country. Use null if truly missing; do not infer. Note: "FYI after Tuesday's call w/ BlueHarbor—actually the invoices still come from BlueHarbor Metrics LLC not the parent BlueHarbor Labs Inc. . Please stop using ap@old-lattice.co mailto:ap@old-lattice.co after June; new billing contact is billing+bhm@latticeworks.io mailto:billing+bhm@latticeworks.io . Contract ref BHM-27A-9941 / renewal dated 2025-08-01. Pricing unchanged except annual cap now US$148,500 all-in. Standard off-ramp stays 45 days written notice, but there is NO termination for convenience during the initial term. Support is Enterprise Plus; they quoted 99.95% monthly uptime SLA. Security paperwork: DPA countersigned on 2025/05/14, security issues should go to soc@blueharbormetrics.com mailto:soc@blueharbormetrics.com . Their tax residency remains Ireland. Ignore my earlier draft mentioning 30 days and the ops alias." Winner: DeepSeek-V4-Pro — Model A cleanly returns just the requested JSON and preserves the note’s facts well, including representing the DPA field with the actual signed date present in the source. Model B adds extra explanatory text despite the formatting instruction and is less accurate on key fields, notably setting vendor name to the legal entity and reducing dpa signed to a boolean instead of the explicit signed date available in the note. Second judge pass, order swapped — scores are the average of both: Model B is better because its extracted values are consistently aligned with the note, especially treating dpa signed as a boolean, while Model A incorrectly outputs a date for that field. Model A is slightly better on strict JSON-only formatting, but Model B is overall more correct; both have a minor issue around vendor name , with Model B being less clearly justified than the note supports. 12. LRU cache Implement a class LRUCache