Kimi K3 wins 6–3

Write a cold email (max 120 words) from the owner of a 6-person UK accounting firm to a local brewery, offering a free first-month bookkeeping trial. Friendly but professional, one clear call to action, no buzzwords.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet constraints, friendly/professional, under 120 words, clear CTA, no buzzwords. Response 1 has more specific brewery detail (duty, taproom payroll) adding credibility and personal touch, giving it a slight edge in usefulness and specificity. / Both meet word limit, are friendly, professional, with clear CTAs and no buzzwords. Response 2 is slightly more specific and personable, but both are equally effective overall. | google/gemini-3.1-pro-preview: A — Both responses follow all instructions and constraints perfectly. Response 1 is slightly better as it includes specific, relevant details for a UK brewery (alcohol duty, taproom payroll) and has a very natural, friendly tone. / Both responses follow all instructions and constraints perfectly. Response 2 is slightly better because it includes specific brewery-related accounting terms (alcohol duty, taproom payroll), making it more tailored and effective as a cold email. | x-ai/grok-4.5: A — Both under 120 words, meet all constraints with clear CTA and no buzzwords. R1 more tailored to brewery/UK specifics and useful; R2 solid but generic. / Both meet max 120 words, one CTA, no buzzwords, friendly-pro tone. R2 more tailored to brewery ops and slightly clearer value, higher usefulness.

Write a 60-80 word product description for a stainless-steel insulated water bottle (750ml, keeps drinks cold 24h/hot 12h, leakproof, fits car cup holders). Target audience: gym-goers. Avoid cliches like 'stay hydrated in style'.

anthropic/claude-sonnet-5: A — Response 1 stays within the 60-80 word range (71 words) and avoids cliches with vivid gym-specific details. Response 2 is only 59 words, violating the word count constraint, and is more generic/less engaging despite being accurate and clear. / Response 1 is 59 words, missing the 60-80 word requirement. Response 2 is 72 words, fits range, avoids clichés, vivid and gym-specific. Both accurate and well-written, but Response 2 better follows constraints and is more engaging and specific. | google/gemini-3.1-pro-preview: A — Response 1 perfectly meets the word count constraint (72 words) and delivers a highly engaging, gym-focused description without cliches. Response 2 falls slightly short of the minimum word count constraint (59 words) and is a bit more generic. / Response 1 is 59 words, violating the 60-80 word constraint. Response 2 is 72 words, meeting the constraint. Response 2 also has a much better tone tailored specifically to the gym-goer target audience. | x-ai/grok-4.5: A — Resp1 meets 60-80 words, hits all specs vividly for gym audience without cliches. Resp2 under word count (59), solid but less engaging. / R1 under word count at 59; both accurate, gym-targeted, cliche-free. R2 more vivid and useful.

Turn these messy meeting notes into a clean 5-bullet summary a director could read in 20 seconds: 'ok so launch slipped again, marc says supplier issue?? maybe 3 wks. sarah wants to push the email campaign anyway but tom thinks wait. budget - we're 8k over, mostly the packaging redo. Q: do we tell retail partners now or after new date confirmed. also NEED to hire the warehouse temp before august rush. next mtg tues.'

anthropic/claude-sonnet-5: A — Both accurate and complete, exactly 5 bullets. Response 1 is more concise, matching the 20-second-read requirement better; Response 2 is slightly wordier, reducing quick scannability. / Both accurate, follow 5-bullet format, and are clear. Response 2 is more concise and scannable in 20 seconds, retaining key details like 'per Marc' while being tighter overall. | google/gemini-3.1-pro-preview: A — Both responses accurately summarize the notes into 5 bullets. Response 1 is slightly punchier and more concise, making it better suited for a quick 20-second read by a busy director. / Both responses accurately summarize the notes into exactly 5 bullets. Response 2 is slightly more concise and punchy, making it better suited for a busy director to read in under 20 seconds. | x-ai/grok-4.5: tie (flipped on swap) — Both accurately capture all key points in exactly 5 clean bullets. Response 1 is tighter and more scannable for a 20-second director read; Response 2 is slightly wordier. / Both deliver accurate 5-bullet summaries covering all notes cleanly and usefully for a director. Equal clarity and fidelity; minor style differences only.

Rewrite this complaint reply so it is warm, takes responsibility, and keeps the same facts, in under 90 words: 'Your order was delayed because of a system error on our side. The refund has been processed and will arrive in 3-5 days. We cannot offer further compensation. Let us know if you have questions.'

anthropic/claude-sonnet-5: B — Both meet word limit and facts, but R2 is cleaner, warmer, and concise without extra meta-commentary. R1 includes unnecessary offer to adjust tone, reducing conciseness and professionalism. / Both meet word limit and facts, but Response 2 adds meta-commentary and a follow-up question, violating conciseness/format expectations. Response 1 is clean, warm, concise, and directly usable. | google/gemini-3.1-pro-preview: B — Both responses follow all instructions and constraints, including the word count. Response 2 is better because it provides only the requested text without unnecessary conversational filler, making it more useful and concise. / Response 1 directly provides the rewritten text without unnecessary conversational filler. It perfectly meets all constraints, maintains the facts, and adopts a warm, responsible tone within the word limit. Response 2 includes unwanted conversational text. | x-ai/grok-4.5: B — Both keep facts, are warm, take responsibility, and stay under 90 words. R2 is cleaner and more directly usable. R1 adds unnecessary meta-commentary and offer to revise, reducing concision and professionalism. / Both keep facts, stay warm, own the error, and under 90 words. R1 is pure and concise. R2 adds meta text and offer, reducing clarity for a busy pro.

Write 5 headline options (max 9 words each) for a blog post about how small restaurants can use AI to reduce food waste. Mix: 2 practical, 2 curiosity-driven, 1 with a number.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet constraints and word limits. Response 1 clearly labels categories matching the mix requirement, aiding clarity. Response 2 lacks category labels, making the required mix less explicit, though content is fine. Response 1 also offers helpful follow-up. / Both meet word limits and category mix, but Response 1 is more concise without unnecessary extra text or follow-up offer, making it cleaner and more useful for a busy professional. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses follow all instructions and constraints perfectly. Response 1 is slightly better because it explicitly labels the categories (practical, curiosity, number), making it easier for the user to see that the mix was achieved. / Both responses followed all instructions and constraints perfectly. Response 1 is slightly better as it provides just the requested headlines without unnecessary conversational filler, making it more concise and useful for a busy professional. | x-ai/grok-4.5: tie (flipped on swap) — Both meet all constraints with valid mix and word counts. R1 clearer via labels and slightly more useful; R2 concise but unlabeled. / Both meet mix and word limits exactly. R1 is cleaner and more concise with no extras. R2 adds unrequested labels and offer, slightly less precise to brief.

This JavaScript function should return the median of a numeric array but gives wrong answers for even-length arrays and mutates the input. Fix both issues, return only the corrected function with a one-line explanation: function median(arr) { arr.sort(); const mid = Math.floor(arr.length / 2); return arr[mid]; }

anthropic/claude-sonnet-5: tie — Both responses correctly fix the mutation and even-length median bugs identically, with a one-line explanation as requested. Response 1 uses markdown formatting while Response 2 uses a code comment; both are equally correct and concise. / Both correctly fix mutation and even-length median bugs identically. Response 1 has a concise one-line explanation as requested; Response 2's explanation is slightly longer but still brief and clear. Both equally correct and useful. | google/gemini-3.1-pro-preview: B — Response 2 followed the 'return only' constraint better by providing just the code and a one-line comment, whereas Response 1 included markdown blocks and extra conversational text. / Response 1 strictly follows the constraint to return ONLY the corrected function and a one-line explanation, using a code comment. Response 2 includes markdown formatting and extra text. | x-ai/grok-4.5: B — Both correctly fix mutation via copy+numeric sort and even-length average. R2 strictly returns only function + one-line comment as required; R1 adds markdown/extra text. / Both fix mutation via copy+numeric sort and even-length avg correctly. R1 alone returns solely the fn + exact one-line expl as required; R2 adds markdown/extra text.

Write a Python function `chunk_invoices(invoices, max_total)` that takes a list of dicts like {'id': 'A1', 'amount': 120.5} and groups them into batches where each batch's summed amount does not exceed max_total. A single invoice larger than max_total goes in its own batch. Preserve order. Include 3 test cases using assert.

anthropic/claude-sonnet-5: tie — Both implementations correctly handle grouping, oversized invoices, order preservation, and include 3 valid assert-based tests. Both are correct and clear; Response 1 has slightly more explanation but both are functionally equivalent and equally good. / Both correctly implement the function with proper edge case handling and include 3 valid assert-based tests. Response 2 has cleaner logic (no special-case branch for oversized invoices) and includes sum-verification, slightly more elegant, but both are correct and clear. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses correctly implement the function and include 3 test cases using assert. Response 1's implementation is slightly more elegant, handling the oversized invoice condition naturally without needing a separate branch. / Both responses provide correct, efficient Python functions that perfectly meet all requirements. Response 1's explicit handling of oversized invoices is slightly easier to read, while Response 2's implicit handling is more elegant. Both include the required 3 assert-based test cases. | x-ai/grok-4.5: tie — Both correctly implement greedy batching preserving order and oversized singles, with 3 valid asserts. Equally clear, useful, and complete. / Both correctly implement greedy batching with oversized handling, order preservation, and exactly 3 assert-based tests. Equally clear, useful, and compliant.

Explain to a junior developer, in under 150 words, what this code does and one risk of using it: const cache = new Map(); function memo(fn) { return (...args) => { const k = JSON.stringify(args); if (!cache.has(k)) cache.set(k, fn(...args)); return cache.get(k); }; }

anthropic/claude-sonnet-5: A — Both correctly explain memoization within word limit. Response 1 identifies a more critical, subtle bug (shared cache across functions) with a fix; Response 2 notes a valid but more generic risk (unbounded memory growth). Both clear and concise. / Both are clear and under 150 words. Response 2 identifies a more critical and accurate risk (shared cache across different functions causing wrong results), while Response 1's memory-growth risk is valid but less severe/insightful. | google/gemini-3.1-pro-preview: A — Both responses accurately explain the code and stay under the word limit. Response 1 is slightly better because it identifies a critical bug in the specific implementation (the shared global cache causing collisions between different functions), which is highly valuable for a junior developer to understand. / Both responses follow all instructions and are under the word limit. Response 2 is superior because it identifies a critical, immediate bug in the provided code (shared cache causing collisions between different functions), whereas Response 1 only mentions a general memory leak risk. | x-ai/grok-4.5: A — Both accurate, under 150 words, clear for juniors. R1 better: identifies critical shared-cache bug + fix; R2's memory risk valid but less severe. / Both accurate, under 150 words, clear for juniors. R2's shared-cache risk is subtler/correcter with fix; R1's leak risk also valid but less critical.

Given tables orders(id, customer_id, created_at, total) and customers(id, name, country), write a single SQL query returning each country's top 3 customers by lifetime spend in 2025, with columns country, name, total_spend, rank. Use a window function. Standard PostgreSQL.

anthropic/claude-sonnet-5: tie — Both queries are correct, use window functions properly, and meet requirements. Nearly identical logic and structure with minor stylistic differences. Response 1 adds a helpful note about tie-breaking, slightly increasing usefulness. / Both queries are essentially identical, correct, use window function, meet requirements. Response 2 adds a helpful tie-break note. Minor stylistic differences only, no substantive advantage either way. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses provide accurate, well-structured PostgreSQL queries that perfectly follow all instructions. They correctly use CTEs, window functions, and filtering to achieve the desired result. Response 1 includes a helpful note about handling ties. / Both responses provide accurate and well-structured SQL queries that meet all requirements. Response 2 is slightly better as it includes a helpful note explaining the behavior of RANK() with ties and how to handle them if strict row limits are needed. | x-ai/grok-4.5: tie (flipped on swap) — Both correct PostgreSQL with window RANK for top-3 2025 spenders by country. R1 slightly better ordered, clearer note on ties; both follow specs fully. / Both are correct PostgreSQL using RANK() window for top-3 2025 spenders per country. Equivalent logic, clear, useful; minor style diffs only.

Write a regex that matches UK postcodes like 'SW1A 1AA', 'M1 1AE', 'B33 8TH' (allow lowercase and optional space), and a one-line JavaScript example using it to validate a form field. Briefly note one edge case your regex does NOT handle.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet requirements. Response 1 is clearer, more concise, and explicitly notes a real regex limitation. Response 2's regex is more precise per official spec but its edge case note (BFPO) is less relevant to regex limitations itself. / Both meet requirements. Response 1 uses full official regex including GIR 0AA, more accurate. Response 2 is simpler but explicitly rejects GIR 0AA, a known valid postcode, which it admits as limitation rather than fixing. | google/gemini-3.1-pro-preview: A — Response 1 provides a simpler, easier-to-understand regex, strictly follows the one-line JavaScript constraint by inlining the regex, and clearly explains a relevant edge case. Response 2 uses a highly complex regex and splits the JavaScript into two lines. / Response 2 provides a simpler, more readable regex suitable for general validation, strictly uses a single-line JavaScript example, and clearly explains a highly relevant edge case. Response 1 is also good but uses a massive, complex regex and splits the JS into two lines. | x-ai/grok-4.5: B — Both match examples and meet all task parts with solid JS. R2's regex is stricter/more complete for real UK formats; R1 is simpler but notes key limits well. R2 slightly clearer overall. / Resp1's regex is more complete/accurate (incl. GIR); both meet format/JS/edge-case reqs clearly. Resp2 simpler but less correct on valid patterns.

A bakery sells loaves at £3.20. Ingredients cost £1.10/loaf, labour £0.90/loaf, fixed costs £480/week. They sell 450 loaves/week. A supplier offers ingredients at £0.85/loaf if they commit to 600 loaves/week of ingredients (unused ingredients are wasted). Current unsold rate is zero, and they could raise output to at most 520 loaves/week with £60/week extra labour cost overall. Should they take the deal? Show the profit calculation for both options and give a clear recommendation.

anthropic/claude-sonnet-5: tie — Both responses are nearly identical in calculations, correctness, and recommendation quality. Both correctly compute break-even at ~483 loaves and give clear recommendations. Response 2 is slightly more detailed but both are equally clear and accurate. / Both responses compute identical figures correctly, provide clear tables, and give consistent recommendations with break-even analysis. Nearly identical quality, clarity, and correctness; no constraint violations. Response 1 slightly more detailed but both equally useful. | google/gemini-3.1-pro-preview: tie — Both responses correctly calculate the profits for both scenarios, clearly present the math, and provide a sound recommendation with a break-even analysis. They are equally useful and accurate. / Both responses correctly calculate the profits for both scenarios and provide a clear, nuanced recommendation. Response 2's use of tables makes it slightly easier to read, but both are excellent and fully address the prompt. | x-ai/grok-4.5: tie — Both correctly compute profits (£60 vs £146), note waste/breakeven ~483, and give conditional recommendation. Equally clear, useful, and complete; no constraints broken. / Both correctly compute profits (£60 vs £146), note waste and breakeven (~483), and give clear conditional recommendation. Equally accurate, complete, clear and useful.

Plan the launch week for a small online course (already recorded). Resources: one founder, a part-time VA (10h), email list of 2,000, £300 ad budget. Produce a 7-day plan, one line per day, each line naming the owner. Flag the single riskiest dependency.

anthropic/claude-sonnet-5: A — Both meet format constraints with per-day owner lines. Response 1 is more detailed, ties VA hours to specific budget use logically, and gives a clearer, more actionable risk flag. Response 2 also solid, tracks ad budget precisely to £300, slightly less strategic depth. / Both meet format (one line/day, owners named, risk flagged). Response 2 offers richer specifics, quantified thresholds, and clearer risk rationale, slightly better usefulness and clarity despite length. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Response 1 provides a more realistic and standard launch sequence (cart opens Day 1, closes Day 7) and accurately tracks the VA's 10 hours. Response 2 opens the cart on Day 3, which is less optimal for a 7-day launch week. / Response 1 strictly follows the formatting constraints, explicitly accounts for the entire ad budget and VA hours, and provides a highly actionable, concise plan. Response 2 includes an unnecessary assumption paragraph and doesn't fully allocate the ad budget. | x-ai/grok-4.5: B — R2 strictly follows one-line-per-day format with owners named; R1 uses multi-line days plus pre-assumption. Both solid plans and risk flags, but R2 clearer/more concise for busy pro. / R1 strictly meets one-line-per-day format with owners named and clear risk flag. R2 adds pre-day assumption and multi-sentence days, violating the one-line constraint, though content is solid.

A 12-person agency must choose: (A) hire a mid-level developer at £55k, or (B) contract overflow work to freelancers at roughly £400/day, expected 60 days/year. Give a recommendation in under 200 words covering cost, flexibility, quality risk, and one non-obvious factor.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both cover cost, flexibility, quality risk, non-obvious factor within word limit. Response 2 adds a sharper UK-specific non-obvious factor (IR35) and is more concise/scannable, giving it a slight edge in usefulness and clarity. / Both correctly analyze cost, flexibility, quality risk, and a non-obvious factor within word limits, well-organized and useful. Response 2 adds a slightly more actionable caveat and billable-cost insight, but both are comparably strong and concise. | google/gemini-3.1-pro-preview: A — Both responses perfectly follow instructions and constraints. Response 1 wins slightly because its non-obvious factor (margin/cost-of-sale vs overhead) is an exceptional business insight for a small agency, whereas Response 2's IR35 point, while valid, is more of a standard compliance issue. / Both responses follow all instructions perfectly and stay under the 200-word limit. Response 2 wins because its non-obvious factor (markup on cost-of-sale vs overhead) is exceptionally insightful and useful for an agency owner. | x-ai/grok-4.5: tie (flipped on swap) — Both under 200 words, cover all four required topics accurately with sound UK-relevant analysis, clear structure, and practical caveats. Equally useful and concise for a busy professional. / Both accurate, cover all required points under 200 words, clear and useful. Response 2 edges ahead with stronger non-obvious factor (billable markup) and practical revisit trigger.

A tank holds 2,400 litres. Pump A fills at 40 L/min. Pump B drains at 25 L/min. A runs from 09:00. B accidentally switches on at 09:20. At 10:00 B is switched off. At what time is the tank full? Show your working.

anthropic/claude-sonnet-5: tie — Both responses correctly compute identical results with clear, correct step-by-step working. No constraints violated. Response 1 uses better formatted headers; equally clear otherwise. Essentially equivalent quality. / Both responses correctly compute the same answer (10:25) with identical, clear working. Response 2 is slightly more formatted but functionally equivalent in correctness and clarity. | google/gemini-3.1-pro-preview: tie — Both responses correctly solve the math problem step-by-step, arriving at the correct final answer of 10:25. Both are clear, concise, and easy to follow. / Both responses correctly calculate the final time as 10:25 and show clear, step-by-step working. Response 1 is slightly more conversational, while Response 2 uses structured headers, but both are excellent and perfectly address the prompt. | x-ai/grok-4.5: tie — Both correctly compute phases and reach 10:25 with clear working. No constraints violated. Equally accurate, complete, and useful. / Both correctly compute phases and reach 10:25 with clear working. No checkable constraints violated. Equally accurate, complete, and useful.

Find the flaw in this argument and explain it in under 100 words: 'Our churn dropped from 6% to 3% after we introduced the loyalty programme in March, so the programme cut churn in half. We should double the loyalty budget to cut churn to 1.5%.'

anthropic/claude-sonnet-5: A — Both correctly identify causation/linearity flaws under 100 words. Response 1 is slightly more structured and actionable, offering concrete suggestions (control group test, modeling returns), making it marginally more useful. / Both correctly identify correlation/causation and linear-scaling flaws within word limit. Response 2 is more structured, clearer, and adds actionable suggestion (test causality), slightly more useful and precise than Response 1's more general treatment. | google/gemini-3.1-pro-preview: A — Both responses correctly identify the two main flaws (correlation vs. causation and assuming linear returns) while staying under the 100-word limit. Response 1 is slightly better structured and provides a useful, actionable recommendation for a business professional. / Both responses correctly identify the flaws (correlation vs causation, and linear scaling fallacy) and stay under the 100-word limit. Response 2 is slightly better structured with bolding and provides a clearer explanation of diminishing returns, making it more useful for a non-technical professional. | x-ai/grok-4.5: tie (flipped on swap) — Both correctly identify correlation/causation and non-linearity flaws. R1 exceeds 100-word limit (~105 words); R2 complies (~78). R2 equally clear, useful, concise for busy pro. / Both correctly identify correlation vs causation and non-linear scaling flaws, stay under 100 words, and are clear/useful for non-experts. Nearly identical quality.

Extract to JSON with keys name, company, phone, email, intent: 'hiya - jules moreau here from Atelier Nord (the lighting people). best number is 07700 900 457, or jules@ateliernord.fr. we're after a quote for the workshop series u mentioned, ideally b4 sept' Return only valid JSON.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly extract all fields into valid JSON with accurate data. Response 1 is more readable with formatting; Response 2 is more concise. Both fulfill the task equally well in substance. / Both correct and complete; task said 'return only valid JSON' - response 1 is compact single-line JSON, arguably cleaner. Response 2 uses multi-line formatting, still valid JSON. Minor preference for conciseness favors response 1. | google/gemini-3.1-pro-preview: tie — Both responses perfectly follow the instructions, extracting the correct information into valid JSON with the requested keys. Response 1 is pretty-printed while Response 2 is minified, but both are equally correct and useful. / Both responses perfectly follow all instructions, accurately extract the required information, and output only valid JSON. Response 2 is pretty-printed, while Response 1 is minified, but both are equally valid and useful. | x-ai/grok-4.5: tie (flipped on swap) — Both extract accurately into valid JSON only. Resp2 more concise; identical content otherwise. No checkable constraints breached. / Both extract all fields accurately into valid JSON only. Minor intent phrasing difference is immaterial; equally correct, compliant, clear and useful.

Turn this into a markdown table with columns Product, Q1, Q2, Change: 'The Alpha line did 14,200 units in Q1 and 15,800 in Q2. Beta slipped from 9,100 to 7,400. The new Gamma launched mid-Q2 with 3,300 units (no Q1 sales). Delta held flat at 5,000 both quarters.' Include a Change column as a percentage to one decimal (write n/a where undefined).

anthropic/claude-sonnet-5: tie — Both responses are identical in content and correctly calculated percentages, differing only in table formatting syntax, which is inconsequential. / Both responses are identical in content, correctly calculated percentages, and follow instructions with n/a for undefined change. Only formatting whitespace differs, which is negligible. | google/gemini-3.1-pro-preview: tie — Both responses perfectly follow all instructions, accurately calculating the percentage changes to one decimal place and correctly using 'n/a' for the undefined change. They both provide valid and identical markdown tables. / Both responses followed all instructions perfectly, accurately calculating the percentage changes to one decimal place and handling the undefined case correctly. They are identical in content. | x-ai/grok-4.5: tie — Both identical correct tables with accurate % changes to 1 decimal and n/a; fully follow format and all specs. / Both produce identical correct markdown tables with accurate one-decimal % changes and n/a; minor alignment syntax differs but irrelevant.

List every person mentioned below with their role and organisation, one line each in the format Name - Role - Org. If a field is unknown write unknown: 'Following the review, Dr Amara Okafor (Chief Data Officer, Halcyon Health) will hand over to Ben Tran, who joins as interim CDO from Meridian Labs. Legal sign-off sits with Priya Shah at Halcyon; the external audit remains with T. Lindqvist.'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both similar, but Response 1 assigns a role to Priya Shah (Legal sign-off) inferred from context, while Response 2 marks it unknown, being less informative. Both share same Ben Tran org ambiguity. / Both correctly list all four people similarly, but Response 1 marks Priya Shah's role as 'unknown' more accurately since her role isn't explicitly named, while Response 2 assumes 'Legal sign-off' as her role, which is less precise. | google/gemini-3.1-pro-preview: A — Both responses follow the formatting instructions perfectly. Response 1 is slightly better because it infers Priya Shah's role from the context ('Legal sign-off'), whereas Response 2 marks it as unknown. / Both responses follow the formatting constraints perfectly. Response 2 is slightly better as it extracts the implied roles (Legal sign-off and External audit) rather than defaulting to 'unknown' for Priya Shah. | x-ai/grok-4.5: tie (flipped on swap) — Resp2 more accurate on Priya role (unknown vs invented); both follow format and extract all four people correctly otherwise. Clear, concise, useful. / R2 correctly extracts Priya's legal role from text; R1 wrongly marks it unknown. Both match required format and cover all four people accurately otherwise.

Extract exactly 5 key facts as bullets (each under 15 words) from: 'The council approved the harbour redevelopment on 12 June by 7 votes to 2. Phase one, costing £4.6m, begins in October and includes the fish market roof. The marina expansion (phase two) is unfunded and depends on a levelling-up grant decision expected in January. Local traders get first refusal on the 14 new retail units. Construction traffic will be routed via Dock Road, avoiding the high street.'

anthropic/claude-sonnet-5: A — Response 1 covers all 5 distinct facts within word limits, including construction traffic routing. Response 2 omits the traffic routing fact, splitting phase one into two bullets instead, missing key info. / Both accurate and concise, but Response 2 covers all 5 distinct facts including construction traffic, while Response 1 omits that fact and splits phase one into two bullets, missing coverage. | google/gemini-3.1-pro-preview: A — Both responses follow all constraints, providing exactly 5 bullets under 15 words each. Response 1 is better because it captures all five distinct facts from the original text, whereas Response 2 splits one fact into two and omits the final point about construction traffic. / Both responses followed all constraints, providing exactly 5 bullets under 15 words each. Response 2 is superior because it captures all the distinct facts from the text, whereas Response 1 splits one fact into two and omits the important information about construction traffic. | x-ai/grok-4.5: A — R1 extracts all 5 accurate key facts concisely under limits. R2 has 5 bullets under limits but adds unstated 'replacing', omits unfunded status and traffic routing, reducing completeness and accuracy. / Both meet exact 5-bullet <15-word format. R1 adds unstated 'replacing' and omits traffic/unfunded details. R2 fully accurate, complete, and more useful.

Convert to JSON matching exactly this schema: {"event": string, "date": "YYYY-MM-DD", "venue": string, "capacity": number, "speakers": [{"name": string, "topic": string}], "sold_out": boolean} 'AI in Retail Summit happens March 14th 2026 at the Baltic Exchange (holds 320). Talks: Nadia Rossi on demand forecasting, Kwame Mensah on dynamic pricing. Tickets still available.' Return only the JSON.

anthropic/claude-sonnet-5: tie — Both responses correctly and identically match the schema with accurate data extraction; only whitespace formatting differs, which is inconsequential. / Both responses correctly and identically match the schema with accurate data, differing only in whitespace formatting. | google/gemini-3.1-pro-preview: tie — Both responses correctly extract the information and format it into the exact JSON schema requested, returning only the JSON. They are functionally identical. / Both responses correctly extract the information, format it exactly according to the provided JSON schema, and return only the JSON as requested. They are functionally identical. | x-ai/grok-4.5: tie — Both outputs are identical valid JSON matching the schema exactly, with correct data extraction and only JSON returned. / Both produce identical correct JSON matching the schema exactly and return only that JSON.

Describe how photosynthesis works in exactly 3 sentences. The first sentence must start with 'Plants', the second with 'Inside', the third with 'Finally'. No sentence may exceed 20 words.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet the 3-sentence structure and word limits. Response 2 is more scientifically precise (mentions ATP, NADPH, Calvin cycle) while still concise, giving it a slight edge in accuracy and usefulness. / Both meet the 3-sentence structure, correct starting words, and word limits (~20 max). Both are factually accurate and clear. Response 1 slightly more precise scientifically (mentions ATP/NADPH, Calvin cycle). Very close quality overall. | google/gemini-3.1-pro-preview: A — Both responses follow all constraints perfectly. Response 1 is better suited for a non-technical professional as it avoids jargon like ATP, NADPH, and the Calvin cycle, making it clearer and more useful. / Both responses follow all constraints perfectly. Response 2 is better suited for a non-technical professional, as it avoids jargon like ATP, NADPH, and the Calvin cycle while remaining factually accurate and clear. | x-ai/grok-4.5: B — Both meet all constraints exactly. R2 more factually precise on stages; R1 simpler but slightly imprecise. Both clear/useful, R2 edges accuracy. / Both meet all constraints exactly with accurate 3-sentence structure. Response 1 more precise on mechanisms (light reactions, Calvin cycle); Response 2 simpler but slightly vaguer. Both clear and useful.

Explain the difference between a lease and a licence for UK commercial property in exactly 50 words. Count carefully - exactly 50.

anthropic/claude-sonnet-5: tie — Both responses are exactly 50 words, accurate, clear, and useful explanations of lease vs licence distinctions. R2 slightly more nuanced with case law hint, R1 slightly clearer for laypersons. Overall very similar quality. / Both responses are exactly 50 words and accurately explain lease vs licence distinctions. Response 2 adds useful statutory context (1954 Act) but is otherwise similar in quality. Both meet constraints and are clear and concise. | google/gemini-3.1-pro-preview: A — Both responses follow the strict 50-word constraint perfectly. Response 1 is slightly better as it explicitly mentions the Landlord and Tenant Act 1954, which is highly relevant to UK commercial property, making it more useful and specific. / Both responses follow the strict 50-word constraint perfectly. Response 2 is slightly better as it explicitly mentions the Landlord and Tenant Act 1954 and security of tenure, which are crucial concepts specifically for UK commercial property. | x-ai/grok-4.5: tie — Both exactly 50 words, factually accurate on UK lease vs licence distinctions, clear and useful. No constraints broken. / Both exactly 50 words, factually accurate on UK lease vs licence distinctions, clear, concise and useful for professionals.

Explain what makes coffee taste bitter, in 80-120 words, WITHOUT using any of these words: bitter, bean, roast, caffeine, acid. Do not use hyphenated or partial variants of them either.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both avoid banned words, explain bitterness factors clearly, within word count (~90-100 words). Similar quality, correctness, and usefulness; Response 2 slightly more detailed but both are comparable in clarity and concision. / Response 2 uses 'roast' variant ('under-extraction'? no) but says 'browning' fine, however uses 'quinic acid' implied via 'quinic' - actually uses 'acid'? No, says 'quinic and phenolic' - safe. But uses 'roast'? No. Response 2 within word count but uses 'stale' fine. Both meet count; Response 1 slightly clearer, no violations detected in either upon recheck. | google/gemini-3.1-pro-preview: B — Both responses successfully avoid the forbidden words and fall within the 80-120 word limit. Response 2 is slightly better as it flows more naturally and is slightly less technical, making it more useful for a busy non-technical professional. / Both responses successfully avoid the forbidden words and meet the length constraints. Response 1 flows better and provides a clearer, more cohesive explanation for a non-technical reader. | x-ai/grok-4.5: tie (flipped on swap) — Both meet word count and ban all forbidden terms/variants. R1 more precise on compounds and extraction balance; R2 solid but slightly vaguer. Equal clarity/usefulness for non-experts. / Both meet word count and ban all forbidden terms/variants with accurate causes. R1 clearer structure and flow for non-experts; R2 slightly denser but solid.

Write a LinkedIn post about hiring your first apprentice. Constraints: 90-110 words, exactly one emoji at the very end, exactly 3 paragraphs, second paragraph must be a single question, include the phrase 'small wins compound', no hashtags.

anthropic/claude-sonnet-5: B — R1 adds extra meta text after the emoji, breaking 'emoji at very end' rule. R2 correctly ends with emoji, meets word count, structure, phrase, and no hashtags. R2 is more compliant and equally useful/clear. / Both meet word count, phrase, emoji, question paragraph. R2 adds an unnecessary meta note breaking the pure post format, effectively violating the clean structure required. R1 is cleaner and fully compliant. | google/gemini-3.1-pro-preview: B — Response 1 included a meta-text note at the end which violated the paragraph count (making it 4), pushed the emoji away from the very end, and pushed the word count over 110. Response 2 followed all constraints perfectly, including the exact word count, paragraph structure, and phrase inclusion. / Response 1 followed all instructions and constraints perfectly. Response 2 included meta-text at the end, which caused it to violate the word count constraint (112 words) and the constraint to have the emoji at the very end. | x-ai/grok-4.5: tie (flipped on swap) — Both meet all constraints exactly (word count, structure, phrase, emoji, no hashtags). R1 is more natural, personal, and polished for LinkedIn; R2 is solid but slightly less fluid with lowercase phrase start. / R1 meets all constraints (106 words, 3 paras, Q as P2, phrase, 1 end emoji, no tags) with minor grammar slip. R2 post complies but trailing note puts emoji not at very end and adds content.

Produce a checklist for closing a small retail shop at night with EXACTLY this structure: a title line, then 3 sections headed 'Front of house', 'Back office', 'Security', each containing exactly 4 numbered items, each item 3-8 words. Nothing else before or after.

anthropic/claude-sonnet-5: tie — Both follow structure exactly with title, three sections, 4 items each, 3-8 words per item. Both accurate and clear, equally useful for a non-technical professional. No meaningful difference in quality. / Both responses follow the exact structure with title, three sections, four items each, all within word limits, and no extra text. Content quality is comparable in usefulness and clarity. | google/gemini-3.1-pro-preview: tie — Both responses followed all instructions perfectly, including the exact structure, section headers, item counts, and word count constraints per item. Both provide highly useful and concise checklists for a retail shop. / Both responses followed all instructions perfectly, including the exact structure, section headers, item counts, and word count constraints per item. Both are highly useful, clear, and concise. | x-ai/grok-4.5: tie — Both responses match the exact required structure, word counts per item, headings, and item numbers with no extras. Content is accurate, useful, and clear for both. / Both match structure, headings, exactly 4 items of 3-8 words each, nothing extra. Content equally accurate, useful, clear.

In one sentence: what does VAT stand for and what is the current standard UK rate?

anthropic/claude-sonnet-5: tie — Both responses are identical, correct, concise, and fulfill the one-sentence requirement accurately. / Both responses are identical, correct, concise, and follow the one-sentence instruction exactly. | google/gemini-3.1-pro-preview: tie — Both responses are identical. They correctly state what VAT stands for and the current UK rate in exactly one sentence. / Both responses are identical, perfectly accurate, and follow the one-sentence constraint exactly. | x-ai/grok-4.5: tie — Identical perfect answers: correct facts, one sentence as required, clear and useful. / Both identical, fully correct, one sentence as required, clear and useful.

Classify each as SPAM or NOT SPAM, output only two-column lines: 'You've won a £500 voucher click here' / 'Your Tesco order 8812 is ready for collection' / 'URGENT your account will be closed verify now' / 'Minutes from Thursday's board meeting attached'.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly classify all items. Response 1 places the label after the text, matching typical expected order; Response 2 reverses column order, which is a minor deviation but still fully valid classification-wise. / Both correctly classify all four items in two-column format as requested; only order of columns differs, task didn't specify order strictly, both equally clear and useful. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses correctly classify the messages and follow the formatting constraint. Response 1 is slightly better as it places the text first and the classification second, which is the standard convention for this type of task. / Both responses correctly classify the messages and perfectly follow the formatting constraint of outputting only two-column lines. | x-ai/grok-4.5: tie (flipped on swap) — Both correctly classify all 4 messages and use only two-column lines. Resp1 matches listed message-first order more naturally; Resp2 reverses columns. / Both correctly classify all four messages and output only two-column lines. Column order differs but task unspecified; equal on all criteria.

Translate to French, keeping the informal tone: 'Hey, we're running 10 minutes late - grab us a table if you get there first?'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both accurately translate with informal tone. Response 2 is more concise as requested implicitly by task simplicity; Response 1 adds useful extra notes but is longer. Both correct and natural French. / Both accurate translations with informal tone. Response 1 is concise and correct; Response 2 has minor grammar issue ('t'arrives' should be 's'arrives' - wrong contraction of 'tu arrives') but offers useful extra context. Task only asked for translation, so extra notes are unnecessary but not harmful. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both provide accurate, informal translations. Response 1 includes helpful context on the informal choices and offers an even more casual alternative, making it slightly more useful, though Response 2 is more concise. / Response 1 provides a direct, accurate, and perfectly informal translation without unnecessary fluff, making it ideal for a busy professional. Response 2 includes good notes but is less concise. | x-ai/grok-4.5: B — Both accurate informal translations. R2 more concise and directly fulfills the pure translation task; R1 adds helpful but unrequested notes reducing focus. / Both accurate informal translations. R1 is direct and concise. R2 adds unrequested notes and alternatives, reducing concision and usefulness for a busy user.

Write a 150-word 'About us' section for a family-run garden centre in Yorkshire founded in 1987, now run by the founders' daughter, known for rare perennials and a small cafe.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet content requirements well, but Response 1 includes an odd meta-note inviting name substitution, which is unnecessary/unprofessional. Response 2 is cleaner, on-brand, and doesn't break the word count as noticeably. Neither strictly hits exactly 150 words, but Response 1's meta comment is a clearer flaw. / Both exceed ~150 words (R1 ~175, R2 ~190) violating word count. R2 invents specific names, which could be seen as fabricating facts not given, but adds charm and includes a disclaimer. R1 is safer, factually accurate, well-structured. Both equally usable and clear. | google/gemini-3.1-pro-preview: B — Response 2 followed the exact word count constraint (150 words) while including all required details beautifully. Response 1 exceeded the word count. / Response 1 perfectly follows all instructions, including the strict 150-word limit, and includes all required details in a well-written, professional tone. Response 2 fails the word count constraint. | x-ai/grok-4.5: tie (flipped on swap) — Both exceed 150 words (156 and 163). Both cover all required elements accurately and clearly for non-experts; 2 is slightly tighter and more professional without invented names. / R1 hits exactly 150 words and all required facts cleanly. R2 exceeds 150 words, invents names, adds title/note, so breaches word-count constraint while remaining otherwise solid.

Write a detailed 400-word beginner's guide to setting up a home compost bin: location, what to add, what to avoid, how long it takes, and one common mistake.

anthropic/claude-sonnet-5: A — Both cover all required elements accurately and clearly. Response 1 uses headers for readability and explicitly labels the 'one common mistake,' better matching the prompt's structure. Response 2 is solid but slightly denser and less clearly organized around the requested points. / Both accurate and cover all required elements. Response 2 uses clear headers, better organization, and readability for a non-technical audience while remaining concise. Response 1 is a solid dense paragraph format, slightly harder to scan. Both likely near 400 words; no explicit violations detected. | google/gemini-3.1-pro-preview: tie — Both responses fail to meet the exact 400-word count constraint (Response 1 is 391 words, Response 2 is 407 words). However, Response 1 is much better structured with clear headings, making it highly useful and easy to read for a busy non-technical professional. / Both responses fail the exact 400-word count constraint. Response 2 is better formatted with clear headings and bullet points, making it much easier for a busy non-technical professional to read and extract the necessary information. | x-ai/grok-4.5: tie (flipped on swap) — Both cover all required topics accurately and usefully at ~400 words. R1 superior in structure, headings, and clarity for non-experts. / Both exceed 400 words and cover all required topics accurately. R2's headings improve structure, clarity, and usefulness for busy readers over R1's dense prose.