Dead heat, 4–4

Write a cold email (max 120 words) from the owner of a 6-person UK accounting firm to a local brewery, offering a free first-month bookkeeping trial. Friendly but professional, one clear call to action, no buzzwords.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet word limit, friendly professional tone, clear CTA, no buzzwords. Response 1 mentions specific brewery-relevant tasks adding usefulness; Response 2 explicitly states word count, mentions team size per no requirement but fine. Both very similar quality. / Both meet word limit, tone, and CTA requirements. Response 2 has a stronger, more specific CTA (15-minute call) and brewery-relevant details, making it slightly more useful and concrete than Response 1's generic reply prompt. | google/gemini-3.1-pro-preview: B — Both responses follow the constraints well and are under the 120-word limit. Response 2 explicitly includes the '6-person' firm detail from the prompt, whereas Response 1 only mentions being a 'small' firm. Response 2 is slightly more concise and direct. / Response 1 includes the specific detail of being a 6-person firm as requested, whereas Response 2 just says 'small'. Both follow the word count constraint and provide a clear, professional email with a single call to action. | deepseek/deepseek-v4-pro: tie (flipped on swap) — Both emails meet all constraints: under 120 words, no buzzwords, one clear CTA, friendly yet professional. Response 1 tailors to breweries; Response 2 is more direct. Equally effective. / Response 1 is more concise, has a lower-friction CTA (just reply), and precisely states 'six-person firm' as required. Response 2 is slightly wordier and its CTA (15-minute call) is a bigger ask. Both are accurate, friendly, and buzzword-free.

Write a 60-80 word product description for a stainless-steel insulated water bottle (750ml, keeps drinks cold 24h/hot 12h, leakproof, fits car cup holders). Target audience: gym-goers. Avoid cliches like 'stay hydrated in style'.

anthropic/claude-sonnet-5: B — R1 is 59 words, missing the 60-80 word requirement; R2 hits 70 words correctly. Both avoid cliches and cover features accurately, but R2 fully complies and is equally clear and useful. / Both are accurate, clear, and avoid cliches. R1 meets the 60-80 word count (70 words), while R2 falls short at 59 words, violating the explicit word count constraint. R1 is slightly more detailed and complete. | google/gemini-3.1-pro-preview: B — Response 1 falls just short of the 60-word minimum constraint (59 words). Response 2 meets the word count constraint (70 words), includes all required features, targets the correct audience, and avoids the banned cliché. / Response 1 followed all instructions, including the 60-80 word count constraint (70 words). Response 2 missed the minimum word count constraint (59 words). Both effectively targeted gym-goers and included all required features without using cliches. | deepseek/deepseek-v4-pro: B — Response 1 word count 59 (below required 60-80), violates constraint. Response 2 meets all criteria, accurate, avoids cliché, targets gym-goers, clear. Winner: 2. / Response 1 meets all constraints, accurate, clear, useful. Response 2 is under word count (59 words), violating the 60-80 word requirement. Both avoid cliches and target gym-goers well. Winner: 1.

Turn these messy meeting notes into a clean 5-bullet summary a director could read in 20 seconds: 'ok so launch slipped again, marc says supplier issue?? maybe 3 wks. sarah wants to push the email campaign anyway but tom thinks wait. budget - we're 8k over, mostly the packaging redo. Q: do we tell retail partners now or after new date confirmed. also NEED to hire the warehouse temp before august rush. next mtg tues.'

anthropic/claude-sonnet-5: tie — Both are accurate, concise 5-bullet summaries covering all key points within instructions. Response 1 slightly more polished formatting, Response 2 slightly more concise. Equally usable for a director. / Both accurately capture all key points in 5 clear bullets, meeting the 20-second-read goal. Response 2 is slightly more polished with bold emphasis but marginally longer/denser. Response 1 is more concise. Both fully satisfy the task's implicit constraints. | google/gemini-3.1-pro-preview: B — Both responses accurately capture the meeting notes in exactly 5 bullets. Response 2 is slightly punchier and more concise, making it slightly better suited for a quick 20-second read by a busy director. / Both responses accurately summarize the notes into exactly 5 bullets. Response 1 is slightly more concise and punchy, making it better suited for a busy director to read in under 20 seconds. | deepseek/deepseek-v4-pro: tie — Both accurately capture all key points in exactly 5 clean, scannable bullets. Response 1 is slightly more formal, Response 2 more concise; both are highly useful and clear. / Both accurately capture all facts, exactly follow the 5-bullet instruction, are clear and concise, and useful for a director. No constraints violated.

Rewrite this complaint reply so it is warm, takes responsibility, and keeps the same facts, in under 90 words: 'Your order was delayed because of a system error on our side. The refund has been processed and will arrive in 3-5 days. We cannot offer further compensation. Let us know if you have questions.'

anthropic/claude-sonnet-5: tie — Both responses are warm, accountable, factually consistent, under 90 words, and clear. Response 2 slightly more explicit in taking responsibility, but both are essentially equal in quality and usefulness. / Both responses are warm, accurate, under 90 words, and retain all facts. R2 shows slightly more empathy by acknowledging frustration, but both are equally clear and concise, making them essentially tied in quality. | google/gemini-3.1-pro-preview: B — Both responses follow all instructions and constraints perfectly. Response 2 is slightly better as it explicitly takes responsibility ('that's on us, and we take full responsibility'), making it feel warmer and more accountable. / Both responses follow all instructions and constraints perfectly. Response 1 is slightly better because it explicitly states 'we take full responsibility', which directly addresses the prompt's requirement to take responsibility in a very clear and warm manner. | deepseek/deepseek-v4-pro: B — Response 2 takes explicit responsibility ('we take full responsibility') and is warmer, making it more useful. Response 1 is empathetic but doesn't enhance responsibility. Both are factually accurate and under word limit. / Response 1 explicitly takes responsibility ('that’s on us') and is warmer, while both keep facts, are under 90 words, and are clear. Response 2 is empathetic but less direct in owning the error.

Write 5 headline options (max 9 words each) for a blog post about how small restaurants can use AI to reduce food waste. Mix: 2 practical, 2 curiosity-driven, 1 with a number.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet word limits and category mix. Response 2 explicitly labels categories, aiding clarity and usefulness for a non-technical reader, while Response 1 is slightly less transparent about which is which. / Both meet the 5-headline mix requirement, word counts under 9, and categories (2 practical, 2 curiosity, 1 number). Response 1 labels categories explicitly, aiding clarity; Response 2 is more concise and punchy. Both are correct and useful; quality is comparable. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses followed all instructions perfectly, including word counts and the specific mix of headline types. Response 1 is slightly better as it presents the headlines cleanly without unnecessary labels, making it more concise and ready to use. / Both responses followed all instructions perfectly, including the word count and specific mix of headline types. Response 1 is slightly better because it explicitly labels the categories, making it easier for the user to see that the constraints were met. | deepseek/deepseek-v4-pro: tie — Both responses provide 5 headlines under 9 words with the required mix (2 practical, 2 curiosity-driven, 1 numbered). They are accurate, clear, and useful for a non-technical audience. / Both responses contain 5 headlines, each ≤9 words, with 2 practical, 2 curiosity-driven, and 1 numbered. Factual and useful, with no errors or breaches.

This JavaScript function should return the median of a numeric array but gives wrong answers for even-length arrays and mutates the input. Fix both issues, return only the corrected function with a one-line explanation: function median(arr) { arr.sort(); const mid = Math.floor(arr.length / 2); return arr[mid]; }

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correct and functionally identical, but Response 1 keeps explanation inline as one line per instructions, while Response 2 uses markdown code block and separate explanation line, slightly deviating from 'return only the corrected function with a one-line explanation' format. / Both responses provide identical correct code with proper fixes and a concise one-line explanation, differing only in code block formatting. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses correctly fix the mutation, numeric sorting, and even-length array issues. Response 1 is slightly better as it strictly returns only the code and the explanation as a comment, avoiding markdown formatting which better adheres to the 'return only' constraint. / Both responses correctly fix the mutation, numeric sorting, and even-length array issues. Response 1 is slightly better formatted with markdown code blocks, making it easier to read. | deepseek/deepseek-v4-pro: tie — Both responses fix mutation and even-length median, add numeric sort. No constraint violations. Identical quality. / Both fix mutation and even-length median correctly, provide a concise one-line explanation, and are equally useful.

Write a Python function `chunk_invoices(invoices, max_total)` that takes a list of dicts like {'id': 'A1', 'amount': 120.5} and groups them into batches where each batch's summed amount does not exceed max_total. A single invoice larger than max_total goes in its own batch. Preserve order. Include 3 test cases using assert.

anthropic/claude-sonnet-5: A — Both correctly implement chunking with oversized invoice handling. Response 1 is slightly more explicit about oversized invoice logic and adds an empty-input test, giving marginally better coverage and robustness (max_total validation). / Both are correct and follow instructions. Response 2 explicitly handles oversized invoices in a clearer dedicated branch, includes an empty input edge case, and has slightly more robust validation, making it marginally more thorough and clear. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses provide correct, well-written Python functions that meet all constraints and handle oversized invoices properly. Response 2 has slightly more elegant logic, while Response 1 is more explicit, but both are excellent. / Both responses provide correct and well-tested solutions. Response 1 is slightly preferred for its more concise and elegant logic that naturally handles oversized invoices without needing a special conditional branch. | deepseek/deepseek-v4-pro: A — Response 1 explicitly handles oversized invoices and invalid input, making it more robust and clear. Response 2’s batch splitting works but is less explicit. Both satisfy all constraints and include 3 assert tests. / Both correctly batch invoices and include 3 asserts. Response 2 adds an explicit oversized-invoice branch, validation for max_total<=0, and more detailed docstring, making it slightly more robust and useful for a professional.

Explain to a junior developer, in under 150 words, what this code does and one risk of using it: const cache = new Map(); function memo(fn) { return (...args) => { const k = JSON.stringify(args); if (!cache.has(k)) cache.set(k, fn(...args)); return cache.get(k); }; }

anthropic/claude-sonnet-5: tie (flipped on swap) — Both are concise and under 150 words. Response 2 correctly identifies the more critical bug: the shared global cache causes key collisions across different wrapped functions, a subtle but important risk that Response 1 misses. / Both are accurate and concise, under 150 words. Response 2 adds useful nuance about JSON.stringify limitations and includes an example, slightly more helpful for a junior dev without exceeding length constraints. | google/gemini-3.1-pro-preview: B — Both responses meet the word count constraint and explain the code well. Response 2 is superior because it correctly identifies a critical bug in the provided snippet: the cache is shared across all wrapped functions, leading to key collisions. / Both responses meet all constraints and explain the code well. Response 1 is slightly better because it correctly identifies a critical bug in the provided code: the cache is shared across all wrapped functions, leading to key collisions. | deepseek/deepseek-v4-pro: tie (flipped on swap) — Response 1 explains more clearly with bullet points and an example, better for a junior developer. Both mention more than one risk against the 'one risk' instruction, but that affects both equally. Response 1's extra risk is still a minor deviation, yet overall more informative. / Both responses correctly explain memoization and a risk, are under 150 words, and are clear. Response 1 notes cache collision; Response 2 notes JSON.stringify limitations. Equally useful.

Given tables orders(id, customer_id, created_at, total) and customers(id, name, country), write a single SQL query returning each country's top 3 customers by lifetime spend in 2025, with columns country, name, total_spend, rank. Use a window function. Standard PostgreSQL.

anthropic/claude-sonnet-5: tie — Both queries are correct, use window functions, group by customer id to avoid name collisions, and filter 2025 dates properly. Nearly identical logic and clarity; Response 1 adds secondary name ordering, a minor plus but not required. / Both queries are correct, use window functions properly, follow instructions, and are clear. Nearly identical logic; response 2 adds minor tie-breaking with name, a slight but negligible improvement. | google/gemini-3.1-pro-preview: tie — Both responses provide accurate, well-structured PostgreSQL queries that perfectly meet all requirements, including the use of window functions and correct column outputs. They both correctly group by customer ID to handle potential duplicate names. / Both responses provide accurate, well-structured PostgreSQL queries that perfectly meet all requirements, including the use of window functions and the correct output columns. They both correctly filter for the year 2025. | deepseek/deepseek-v4-pro: tie — Both queries are accurate, use RANK() window function, filter 2025 correctly, and return required columns. Response 1 explicitly casts dates and orders by name for tie-breaking; Response 2 is concise. Both meet all instructions perfectly. No constraints violated. / Both queries correctly compute top 3 per country by 2025 spend with window function. They are accurate, meet all instructions, and are clear. Response 2 adds deterministic tiebreaker ordering, but neither deviates from requirements.

Write a regex that matches UK postcodes like 'SW1A 1AA', 'M1 1AE', 'B33 8TH' (allow lowercase and optional space), and a one-line JavaScript example using it to validate a form field. Briefly note one edge case your regex does NOT handle.

anthropic/claude-sonnet-5: A — Response 1's regex is more accurate to official UK postcode format rules, handles GIR 0AA, and explicitly notes a valid edge case (BFPO). Response 2's regex is looser and its noted edge case is actually handled by their claim inaccurately (GIR 0AA not covered by their regex but not mentioned as such). / Response 2's regex is more accurate to real UK postcode rules and includes GIR 0AA handling, plus a valid edge case note. Response 1's regex is simpler but less precise. Both meet format requirements. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Response 1 provides a highly accurate, standard regex for UK postcodes, whereas Response 2 provides a naive approximation that allows many invalid formats. Both follow all instructions, but Response 1 is much more useful and correct. / Both responses follow all instructions perfectly. Response 1 provides a simpler regex which is easier to read, while Response 2 provides a highly accurate, production-ready regex. Both correctly include the JS one-liner and an edge case. | deepseek/deepseek-v4-pro: tie (flipped on swap) — Response 1's regex precisely follows official UK postcode rules and excludes invalid characters; Response 2 is too permissive, accepting many non-conforming strings. Both satisfy task instructions, but 1 is far more accurate and professionally useful. / Response 1 is simpler, more concise, and equally correct for the given examples, making it more useful for a non-technical professional. Response 2 is more comprehensive but less clear.

A bakery sells loaves at £3.20. Ingredients cost £1.10/loaf, labour £0.90/loaf, fixed costs £480/week. They sell 450 loaves/week. A supplier offers ingredients at £0.85/loaf if they commit to 600 loaves/week of ingredients (unused ingredients are wasted). Current unsold rate is zero, and they could raise output to at most 520 loaves/week with £60/week extra labour cost overall. Should they take the deal? Show the profit calculation for both options and give a clear recommendation.

anthropic/claude-sonnet-5: A — Both compute correctly (£60 vs £146), with response 2 having a minor labour miscalc (528 instead of 468+60=528, actually correct). Response 1 is more detailed, clear tables, caveats on sales feasibility, slightly more thorough recommendation. / Both correctly compute £60 vs £146 profit and recommend the deal contingent on selling 520 loaves. Response 2 is more thorough, clearly formatted, and explicitly addresses the 450-loaf contingency with correct math, making it more useful despite slightly more length. | google/gemini-3.1-pro-preview: A — Both responses correctly calculate the profits for both scenarios. Response 1 is slightly better formatted and provides a more nuanced, business-focused recommendation. / Both models correctly calculate the profits for both scenarios and provide a clear recommendation. Model 2 is slightly better formatted and easier to read for a non-technical professional. | deepseek/deepseek-v4-pro: tie — Both responses correctly calculate profits (£60 current, £146 under deal at 520 loaves), show workings, and give a clear recommendation. No explicit constraints were broken. / Both responses accurately compute current and deal profits, showing £60 vs £146, and recommend taking the deal. No instructions broken.

Plan the launch week for a small online course (already recorded). Resources: one founder, a part-time VA (10h), email list of 2,000, £300 ad budget. Produce a 7-day plan, one line per day, each line naming the owner. Flag the single riskiest dependency.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both provide clear 7-day plans with owners per line and a flagged risk. Response 1 is more detailed but slightly verbose; Response 2 is more concise and structured, with a plausible risk flag. Both meet constraints equally well. / Both meet format with one line per day and named owners, flag a risk. Response 1 is more concise and clearer; Response 2 has denser lines and slightly exceeds VA's 10h budget when summed, a minor inconsistency. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Response 1 is superior because it explicitly tracks the VA's 10 hours to ensure feasibility, and its launch strategy is more realistic (opening the cart on Day 1 rather than Day 3). Both followed all formatting constraints perfectly. / Both responses follow all instructions perfectly, including the constraints of one line per day and naming the owners. Both provide practical, realistic plans that account for the limited resources and budget. Response 1 is slightly more concise, while Response 2 tracks VA hours explicitly. | deepseek/deepseek-v4-pro: tie (flipped on swap) — Response 1 details realistic, timed tasks using exact ad spend and VA hours (total 10h) and correctly flags pre-launch tech readiness as risk. Response 2 has founder finalizing sales page on Day 1, which is unrealistic; it’s less granular and overlooks critical checks. / Both plans meet all constraints: 7 days, one line per day, owner named, risk flagged. Response 1 clearly allocates the £300 ad budget; Response 2 precisely tracks the 10 VA hours. Each is correct, useful, and clear. No factual errors. They are equally strong.

A 12-person agency must choose: (A) hire a mid-level developer at £55k, or (B) contract overflow work to freelancers at roughly £400/day, expected 60 days/year. Give a recommendation in under 200 words covering cost, flexibility, quality risk, and one non-obvious factor.

anthropic/claude-sonnet-5: tie — Both responses are accurate, cover all required elements within word limit, and offer similar quality recommendations with clear structure. Response 2's non-obvious factor is slightly less novel but still valid; both are equally useful and concise. / Both cover cost, flexibility, quality risk, and a non-obvious factor within limits, with sound figures and clear recommendations. Response 2 slightly more nuanced on breakeven point and bottleneck insight, but both are comparable in quality and concision. | google/gemini-3.1-pro-preview: A — Both responses follow all instructions perfectly and are under the 200-word limit. Response 1 provides a slightly more insightful non-obvious factor regarding the bottleneck of senior review in a small agency, making it marginally more useful. / Both responses perfectly follow instructions and constraints. Response 2 wins slightly because its non-obvious factor (senior review bottleneck) is exceptionally insightful for a small agency context, making it slightly more useful. | deepseek/deepseek-v4-pro: tie — Both accurately cover cost, flexibility, quality risk, and a non-obvious factor; clear, concise, useful. No constraint breached. / Both responses are accurate, cover all required aspects, are under 200 words, and are clear and useful. They offer similar recommendations with slightly different non-obvious factors, both valid. No clear winner.

A tank holds 2,400 litres. Pump A fills at 40 L/min. Pump B drains at 25 L/min. A runs from 09:00. B accidentally switches on at 09:20. At 10:00 B is switched off. At what time is the tank full? Show your working.

anthropic/claude-sonnet-5: tie — Both responses are correct, follow identical logic, and arrive at 10:25 with clear working. Response 2 has a minor mislabeling (40×15 instead of 15×40) but result correct. Both equally clear and useful. / Both responses correctly compute identical steps and arrive at 10:25, with clear working. Response 2 explicitly states the empty-tank assumption, slightly improving clarity, but both are essentially equivalent in quality. | google/gemini-3.1-pro-preview: tie — Both responses correctly calculate the time the tank is full, showing clear, step-by-step working. They are equally accurate, concise, and easy to follow. / Both responses correctly solve the math problem, show clear step-by-step working, and arrive at the correct final time of 10:25. Response 2 explicitly states the assumption that the tank starts empty, but both are excellent. | deepseek/deepseek-v4-pro: tie — Both responses are factually correct, show clear step-by-step working, and meet all instructions. They are equally useful and concise. / Both responses correctly calculate 10:25, show clear step-by-step working, are useful to a non-technical professional, and fully meet all task requirements. Neither violates any explicit constraint.

Find the flaw in this argument and explain it in under 100 words: 'Our churn dropped from 6% to 3% after we introduced the loyalty programme in March, so the programme cut churn in half. We should double the loyalty budget to cut churn to 1.5%.'

anthropic/claude-sonnet-5: tie — Both correctly identify correlation/causation issue and non-linear extrapolation flaw, within word limit, clear and concise. Response 2 adds fallacy names, slightly more precise; Response 1 slightly more thorough on confounders. Essentially equal quality. / Both correctly identify correlation/causation and linear extrapolation flaws, within word limit, clear and concise. Response 2 adds a slightly more actionable suggestion (controlled test), Response 1 is more concise with bolded summary. Both are equally strong overall. | google/gemini-3.1-pro-preview: A — Both responses correctly identify the two main flaws (correlation vs. causation and linear extrapolation) and stay well under the 100-word limit. Response 1 is slightly more cohesive and professional in its tone, making it better suited for a business context. / Both responses correctly identify the flaws and stay under the word limit. Response 2 is better suited for a non-technical professional as it avoids academic jargon (like 'post-hoc fallacies') and provides a practical recommendation. | deepseek/deepseek-v4-pro: tie (flipped on swap) — Response 1 clearly explains both flaws without jargon, making it more accessible to non-technical readers. Response 2 is correct but uses technical fallacies names that may hinder understanding. Both adhere to word limit. / Both correctly identify correlation/causation and non-linearity. Response 2 adds practical advice, slightly more useful for non-technical users, but both are accurate and clear. Score tied at 9.

Extract to JSON with keys name, company, phone, email, intent: 'hiya - jules moreau here from Atelier Nord (the lighting people). best number is 07700 900 457, or jules@ateliernord.fr. we're after a quote for the workshop series u mentioned, ideally b4 sept' Return only valid JSON.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly extract fields; Response 1 is on a single line matching 'return only valid JSON' more concisely, while Response 2 uses multi-line formatting. Content is equivalent and accurate. Response 1 slightly more concise/clean. / Both responses correctly extract all fields into valid JSON with equivalent content. Response 2 is slightly more concise (single line, minified) matching 'return only valid JSON' instruction marginally better, but both are essentially equivalent in correctness and usefulness. | google/gemini-3.1-pro-preview: tie — Both responses correctly extract the requested information into valid JSON with the exact keys specified. Neither includes any extra text, perfectly following the negative constraint. / Both responses correctly extract the requested information into valid JSON with the specified keys. They both follow the negative constraint to return only valid JSON. Response 1 is pretty-printed, while Response 2 is minified, but both are perfectly acceptable. | deepseek/deepseek-v4-pro: tie — Both responses correctly extract all fields, follow instructions exactly, and are clear and useful. No constraint violations. / Both correctly extract all fields into valid JSON, follow the instruction precisely, and are equally useful and concise for a non-technical professional. No constraint violations.

Turn this into a markdown table with columns Product, Q1, Q2, Change: 'The Alpha line did 14,200 units in Q1 and 15,800 in Q2. Beta slipped from 9,100 to 7,400. The new Gamma launched mid-Q2 with 3,300 units (no Q1 sales). Delta held flat at 5,000 both quarters.' Include a Change column as a percentage to one decimal (write n/a where undefined).

anthropic/claude-sonnet-5: tie (flipped on swap) — Both nearly identical and correct; Response 1 includes explicit + sign for positive change, slightly clearer formatting for sign consistency with negative values. / Both correctly compute percentages and format tables identically in content; Response 2 adds alignment and a '+' sign, a minor stylistic difference but not required or forbidden. Both fully meet task requirements. | google/gemini-3.1-pro-preview: A — Both responses follow all instructions perfectly and calculate the percentages correctly. Response 1 is slightly better because it right-aligns the numerical columns and includes a plus sign for the positive change, making it easier to read. / Both responses follow all instructions perfectly and calculate the percentages correctly. Response 2 is slightly better because it right-aligns the numerical columns and includes a '+' sign for the positive change, making it easier to read. | deepseek/deepseek-v4-pro: tie — Both tables accurately present the data, compute percentages to one decimal, and use n/a for undefined. They follow all instructions and are equally clear and useful. / Both are factually accurate, follow instructions exactly, present data clearly. No constraint breaches. Identical scores.

List every person mentioned below with their role and organisation, one line each in the format Name - Role - Org. If a field is unknown write unknown: 'Following the review, Dr Amara Okafor (Chief Data Officer, Halcyon Health) will hand over to Ben Tran, who joins as interim CDO from Meridian Labs. Legal sign-off sits with Priya Shah at Halcyon; the external audit remains with T. Lindqvist.'

anthropic/claude-sonnet-5: B — Both correctly list all four people, but Response 2 provides a more accurate role for Priya Shah (Legal sign-off) instead of marking it unknown, better fulfilling the task's intent while remaining concise and correctly formatted. / Both correctly list Okafor, Tran (though Org ambiguous, both same), Lindqvist. Response 1 correctly infers Priya Shah's role as legal-related, while Response 2 marks it unknown despite explicit text stating her legal role, making Response 1 more accurate. | google/gemini-3.1-pro-preview: B — Response 2 accurately captures Priya Shah's role based on the text ('Legal sign-off') and uses the exact organization name provided ('Halcyon'). Response 1 incorrectly lists her role as unknown and infers 'Health' for her organization. / Response 1 correctly identifies Priya Shah's role as 'Legal sign-off' based on the text, whereas Response 2 incorrectly lists it as 'unknown'. | deepseek/deepseek-v4-pro: tie (flipped on swap) — Response 1 correctly uses unknown for Priya Shah's role, but both miss that Ben Tran's organisation is Halcyon Health, not Meridian Labs, and T. Lindqvist's role should be unknown. Response 2 has additional errors: Priya's role is not 'Legal sign-off' and T. Lindqvist's role is not 'external audit'. Response 1 is more accurate and useful. / Response 1 correctly lists all persons except Ben Tran's org (should be unknown, not Meridian Labs) but handles other fields accurately. Response 2 additionally misses Priya Shah's known role, reducing usefulness. Both breach the explicit 'unknown' rule for Ben Tran.

Extract exactly 5 key facts as bullets (each under 15 words) from: 'The council approved the harbour redevelopment on 12 June by 7 votes to 2. Phase one, costing £4.6m, begins in October and includes the fish market roof. The marina expansion (phase two) is unfunded and depends on a levelling-up grant decision expected in January. Local traders get first refusal on the 14 new retail units. Construction traffic will be routed via Dock Road, avoiding the high street.'

anthropic/claude-sonnet-5: tie — Both extract 5 accurate bullets under 15 words each, covering key facts. Response 1 omits Dock Road detail, Response 2 omits it too but mentions phase two explicitly. Both nearly identical quality; minor omission of construction traffic detail in both. / Both extract 5 accurate bullets under 15 words, omitting construction traffic detail but that's acceptable given 5-fact limit. Both equally clear and correct; minor wording differences don't favor either significantly. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses follow all constraints perfectly. Response 2 is better because it captures the 'unfunded' detail of phase two, and Response 1 hallucinates the word 'repairs' which was not in the source text. / Both responses follow all instructions perfectly, extracting exactly 5 facts with each bullet under 15 words. They are both highly accurate, concise, and useful. | deepseek/deepseek-v4-pro: B — Response 1 incorrectly adds 'repairs' to bullet 3, not in source. Response 2 is fully accurate and concise. Both meet word and bullet constraints. / Response 2 adds 'repairs' to bullet 3, inventing detail not in the text, so it is factually inaccurate. Response 1 is fully correct and follows all constraints exactly.

Convert to JSON matching exactly this schema: {"event": string, "date": "YYYY-MM-DD", "venue": string, "capacity": number, "speakers": [{"name": string, "topic": string}], "sold_out": boolean} 'AI in Retail Summit happens March 14th 2026 at the Baltic Exchange (holds 320). Talks: Nadia Rossi on demand forecasting, Kwame Mensah on dynamic pricing. Tickets still available.' Return only the JSON.

anthropic/claude-sonnet-5: tie — Both responses are correct, complete, properly formatted JSON matching the schema exactly, with only whitespace differences. / Both responses are identical in content, correctly matching the schema with accurate data extraction and formatting, differing only in whitespace. | google/gemini-3.1-pro-preview: tie — Both responses perfectly follow the instructions, accurately extract the information, match the required JSON schema exactly, and return only the JSON without any extra text. / Both responses perfectly extract the information into the requested JSON schema and follow the negative constraint to return only the JSON. | deepseek/deepseek-v4-pro: tie — Both JSONs are correct, match schema, all constraints met; only formatting differs. / Both responses extract all details accurately, adhere exactly to the schema, return only JSON without extra text, and are clear and useful for a non-technical professional.

Describe how photosynthesis works in exactly 3 sentences. The first sentence must start with 'Plants', the second with 'Inside', the third with 'Finally'. No sentence may exceed 20 words.

anthropic/claude-sonnet-5: A — Both meet sentence count, starters, and word limits. Response 1 is more accurate and detailed (mentions ATP/NADPH, Calvin cycle), while Response 2 is simpler but less precise. Both clear and concise. / Both meet sentence count, starting words, and word limits. Response 2 is more scientifically detailed and accurate, while still concise and compliant. | google/gemini-3.1-pro-preview: A — Both responses follow all constraints perfectly. Response 1 is much better written, more scientifically accurate, and clearer, whereas Response 2 lacks proper punctuation (e.g., missing commas in a list) making it slightly clunky. / Both responses follow all constraints perfectly. Response 2 is slightly better written with proper punctuation and provides a more accurate, yet accessible, scientific explanation. | deepseek/deepseek-v4-pro: A — Response 1 accurately details light-dependent and light-independent stages with correct terminology, fitting constraints perfectly. Response 2 oversimplifies and contains factual imprecisions, making it less correct and useful. / Response 2 provides a more accurate, detailed, and clear step-by-step explanation of photosynthesis, mentioning light-dependent reactions and the Calvin cycle, which is more useful for a professional. Both followed all constraints.

Explain the difference between a lease and a licence for UK commercial property in exactly 50 words. Count carefully - exactly 50.

anthropic/claude-sonnet-5: tie — Both responses are exactly 50 words, accurate, clear, and cover the key distinctions between leases and licences equally well. / Both responses are exactly 50 words, accurate, clear, and cover the key distinctions between leases and licences equally well. Quality and correctness are essentially equivalent. | google/gemini-3.1-pro-preview: B — Both responses successfully meet the strict 50-word constraint and accurately explain the legal distinction. Response 2 is slightly better as it explicitly references 'UK commercial property' as requested in the prompt, making it more tailored to the specific context. / Both responses successfully meet the strict 50-word constraint and accurately explain the legal distinction. Response 1 is slightly better as it explicitly references 'UK commercial property' as requested in the prompt, whereas Response 2 uses the generic term 'premises'. | deepseek/deepseek-v4-pro: tie (flipped on swap) — Both responses accurately distinguish lease (proprietary interest, exclusive possession) from licence (personal permission) in exactly 50 words. No constraint violations. They are clear and useful to a professional. Scores: 10 each, tie. / Both are factually accurate and exactly 50 words. Response 1 explicitly mentions UK commercial property, better addressing the task context, while Response 2 omits UK. Both are clear and useful, but 1 is slightly more precise.

Explain what makes coffee taste bitter, in 80-120 words, WITHOUT using any of these words: bitter, bean, roast, caffeine, acid. Do not use hyphenated or partial variants of them either.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both stay within word limits (~100 words) and avoid banned words. Both explain over-extraction and heat effects clearly. Response 1 slightly more detailed and useful; Response 2 slightly more concise. Roughly equal quality. / Both meet word count and avoid banned words. Response 2 offers more accurate, detailed explanation of chemical causes while response 1 is slightly vaguer. Both clear and concise, but response 2 is more informative for a non-technical reader. | google/gemini-3.1-pro-preview: A — Both responses perfectly follow the negative constraints and word count limits. Response 1 is slightly better as it addresses both the heating/preparation phase and the brewing phase, providing a more comprehensive explanation of the harsh flavors. / Both responses follow all constraints, including word count and negative constraints. Response 2 provides a more comprehensive explanation by cleverly alluding to the roasting process and overextraction, making it slightly more informative. | deepseek/deepseek-v4-pro: tie (flipped on swap) — 1 covers heat-induced chemical changes, over-extraction, grind, temperature, and staleness, clearly and accurately without forbidden words. 2 is correct but less thorough. Both follow constraints exactly. / Both responses accurately explain bitterness causes (over-extraction, heat, grind) without forbidden words, stay within 80-120 words, and are clear and useful. No constraint violations.

Write a LinkedIn post about hiring your first apprentice. Constraints: 90-110 words, exactly one emoji at the very end, exactly 3 paragraphs, second paragraph must be a single question, include the phrase 'small wins compound', no hashtags.

anthropic/claude-sonnet-5: B — R1 exceeds the 110-word limit (113 words), violating the constraint. R2 meets all constraints: word count, 3 paragraphs, single question, required phrase, no hashtags, one emoji at end. Both are clear and useful, but R2 fully complies. / R1 stays within 90-110 words (106) meeting all constraints. R2 exceeds word limit (113 words), violating the explicit range. Both include required phrase, question paragraph, single ending emoji, no hashtags, but R1 fully complies. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Response 1 failed the word count constraint (113 words). Response 2 followed all instructions perfectly, including the word count, paragraph structure, required phrase, and emoji placement. / Response 1 violates the paragraph constraint by placing the emoji on a new line (creating a 4th paragraph). Response 2 violates the word count constraint (it is over 110 words). Both otherwise follow the formatting and content instructions well. | deepseek/deepseek-v4-pro: B — Response 2 flows better with a more personalized question, and is slightly clearer. Both are useful and follow most constraints, but 1 exceeds the word limit (113 words). / Response 1 perfectly meets all constraints, includes the phrase, and delivers high clarity and usefulness. Response 2 exceeds the 110‑word limit (112 words) and is slightly less concise, but otherwise follows instructions.

Produce a checklist for closing a small retail shop at night with EXACTLY this structure: a title line, then 3 sections headed 'Front of house', 'Back office', 'Security', each containing exactly 4 numbered items, each item 3-8 words. Nothing else before or after.

anthropic/claude-sonnet-5: A — Both meet structure and word-count constraints correctly. Response 1 has clean spacing between sections improving readability, while Response 2 lacks blank lines, making it slightly less clear. Both are accurate and useful. / Both meet structure requirements with correct sections and item counts, word counts within range. Response 2 has better formatting with blank lines improving clarity, and items feel slightly more natural/varied. Response 1's 'Empty trash bins thoroughly' is only 4 words but valid; both are compliant overall. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses followed all constraints perfectly. Response 1 is slightly better because its categorization of tasks is more logical (e.g., locking entrance doors is placed under Security rather than Front of house). / Both responses follow all instructions perfectly, including the exact structure, section headers, item counts, and word count constraints per item. Both provide highly relevant and useful checklists for a retail shop. | deepseek/deepseek-v4-pro: tie — Both responses perfectly match the required structure: title, three sections with exact headings, four items each, all 3-8 words, no extra text. Both are factually correct and useful. / Both responses strictly follow the required structure and word counts. Content is accurate and useful, though each has a minor typo ('Sweep' vs 'Straighten'). They are equally clear and actionable.

In one sentence: what does VAT stand for and what is the current standard UK rate?

anthropic/claude-sonnet-5: tie — Both responses are identical, correct, concise, one-sentence answers meeting the task requirements perfectly. / Both responses are identical, accurate, concise, and fulfill the one-sentence requirement correctly. | google/gemini-3.1-pro-preview: tie — Both responses are identical, perfectly accurate, concise, and follow the one-sentence constraint. / Both responses are identical, perfectly accurate, and follow the single-sentence constraint exactly. | deepseek/deepseek-v4-pro: tie — Both responses are identical, factually correct, and perfectly follow the one-sentence instruction. No constraints violated. / Both responses are identical, factually correct, and follow the one-sentence instruction exactly.

Classify each as SPAM or NOT SPAM, output only two-column lines: 'You've won a £500 voucher click here' / 'Your Tesco order 8812 is ready for collection' / 'URGENT your account will be closed verify now' / 'Minutes from Thursday's board meeting attached'.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly classify all four items. Response 1 uses cleaner two-column format without quotes/slashes; Response 2 retains quotes and slash format, slightly less clean. Both follow instructions reasonably well, but Response 1 is more concise and clear. / Both correctly classify all items. Response 1 uses the original order (item/label) matching the task's implied format more closely, while Response 2 reverses column order. Both are clear and concise; Response 1 slightly better aligned with expected format. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses correctly classify the items and follow the formatting constraint. Response 1 is slightly cleaner by omitting the quotation marks, making it easier to read. / Both responses correctly classify the items and follow the two-column constraint. Response 1 is slightly better as it retains the original quotation marks from the prompt, making it a more exact match to the input text. | deepseek/deepseek-v4-pro: tie (flipped on swap) — Both responses correctly classify all phrases, follow the two-column line format without extra text, and are clear and concise. No constraint violations. / Both classify correctly and use two-column lines. Response 1 puts the phrase first, which is more natural for scanning; Response 2 reverses columns, slightly less intuitive for a busy professional.

Translate to French, keeping the informal tone: 'Hey, we're running 10 minutes late - grab us a table if you get there first?'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both accurate, informal translations. Response 2 uses unnecessary bold formatting and slightly awkward apostrophe contraction 't'arrives'. Response 1 is cleaner and more natural. / Both translations are accurate, informal, and natural French. Response 1 uses casual 't'arrives', response 2 slightly more formal but still informal. Both equally clear and concise; no constraints violated. | google/gemini-3.1-pro-preview: B — Both translations are accurate and maintain an informal tone. Response 2 is slightly better as 't'arrives avant' perfectly captures the casual, conversational feel of a quick text message compared to the slightly more formal 'tu arrives avant nous'. / Response 1 captures the informal tone slightly better by using the spoken contraction 't'arrives', which perfectly matches the casual nature of the original English text. | deepseek/deepseek-v4-pro: tie (flipped on swap) — Both are accurate and informal. Response 2 uses the more colloquial contraction 't’arrives' and omits the redundant 'nous', making it slightly more natural and concise. Response 1 is fully correct but a shade less streamlined. / Both translations are factually correct and keep the informal tone. Response 1 uses a contraction ('t’arrives'), Response 2 is explicit ('tu arrives'). Both meet all instructions, are useful, clear, and concise. No constraints broken.

Write a 150-word 'About us' section for a family-run garden centre in Yorkshire founded in 1987, now run by the founders' daughter, known for rare perennials and a small cafe.

anthropic/claude-sonnet-5: tie (flipped on swap) — Response 1 is ~160 words, close to target; Response 2 is ~200 words, exceeding the 150-word constraint noticeably. Both are accurate and well-written, but Response 1 better follows the word count instruction, making it more compliant. / Both responses meet the brief accurately and clearly, mentioning founding year, daughter's leadership, rare perennials, and cafe. Both are close to 150 words, warm in tone, and useful for a non-technical reader. Quality and style are comparably strong, with no clear differentiator. | google/gemini-3.1-pro-preview: A — Response 1 followed the exact 150-word constraint perfectly and included all requested details in a warm, professional tone. Response 2 failed the word count constraint (approx 175 words). / Response 1 failed the strict 150-word constraint (it is around 180 words). Response 2 hit exactly 150 words while seamlessly incorporating all the requested details in a warm, professional tone. | deepseek/deepseek-v4-pro: A — Response 1 is exactly 150 words, perfectly concise and covers all required elements. Response 2 is longer, less concise, but still accurate and useful. Response 1 wins. / Response 1 exceeds 150 words, violating the constraint. Both are accurate and well-written, but Response 2 meets the word limit exactly while being equally useful and clear.

Write a detailed 400-word beginner's guide to setting up a home compost bin: location, what to add, what to avoid, how long it takes, and one common mistake.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both cover location, additions, avoidances, timeline, and a common mistake accurately and clearly. Response 2 uses headers for better readability; both are near the 400-word target, response1 slightly under likely, response2 claims 412 words. Content quality is comparable, so it's a tie. / Both are accurate, complete, well-structured guides covering all required elements. Response 1 explicitly confirms word count (412) meeting the constraint closely; Response 2 has no count but appears slightly longer and has a grammar error ('A outdoor bin'). Response 1 is clearer and more organized with headers. | google/gemini-3.1-pro-preview: tie — Both responses provide excellent, accurate, and well-structured guides covering all requested topics. However, both fail the strict length constraint of being exactly 400 words long. / Both responses provide excellent, accurate, and useful guides covering all required topics. However, both failed to meet the exact 400-word count constraint (Response 1 is 416 words, Response 2 is 422 words). They tie on quality and usefulness. | deepseek/deepseek-v4-pro: tie (flipped on swap) — Response 1 gives a more precise green-brown ratio and a single, specific common mistake with a fix, making it slightly more accurate and focused. Both are clear and useful, but Response 1 edges out. / Both guides are accurate, cover all required points, and are clear and useful. Response 1 exceeds the 400-word limit, but content quality is equivalent.