Grok 4.5 wins 5–2

Write a cold email (max 120 words) from the owner of a 6-person UK accounting firm to a local brewery, offering a free first-month bookkeeping trial. Friendly but professional, one clear call to action, no buzzwords.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet word limit, tone, CTA, and constraints well. Response 2 is slightly more specific and personalized (VAT, payroll, growth mention), giving marginal edge, but both are strong and comparable in quality. / Both meet word limit and tone. Response 1 is more specific and concise with clear CTA. Response 2 is slightly wordier but still solid, clear CTA. Both avoid buzzwords and forbidden terms. | google/gemini-3.1-pro-preview: B — Both responses follow all instructions and constraints perfectly. Response 2 is slightly better as it mentions 'VAT', which subtly reinforces the UK context requested in the prompt, making it highly relevant and useful. / Both responses follow all instructions perfectly and stay under the word limit. Response 1 is slightly better because it includes a subtle nod to the UK context by mentioning VAT, making it feel more authentic to the prompt's persona. | openai/gpt-5.6-terra: tie (flipped on swap) — Both are under 120 words, friendly and professional, clearly offer a free first-month bookkeeping trial, avoid buzzwords, and include one clear call to action. Response 1 is slightly more brewery-specific; Response 2 adds relevant VAT and payroll context. Both are accurate, concise, and useful. / Both are under 120 words, professional, clear, and include one actionable call to action. Response 1 more explicitly establishes the sender as the owner of a six-person local firm and better reflects UK bookkeeping needs through VAT and payroll. Response 2 is strong but does not explicitly identify the firm as UK-based.

Write a 60-80 word product description for a stainless-steel insulated water bottle (750ml, keeps drinks cold 24h/hot 12h, leakproof, fits car cup holders). Target audience: gym-goers. Avoid cliches like 'stay hydrated in style'.

anthropic/claude-sonnet-5: B — Both meet word count and avoid cliches. Response 2 is slightly more concise, factual, and avoids the somewhat clichéd 'Crush your goals' ending in Response 1, making it marginally better for clarity and tone. / Both meet word count and avoid the banned cliché. Response 1 is concise, professional, and includes all required features clearly. Response 2 is engaging but slightly less concise and uses gym slang ('the box') that may confuse a non-technical audience. | google/gemini-3.1-pro-preview: A — Both responses follow all instructions and constraints, including the word count. Response 1 has a much better marketing tone tailored to the gym-goer audience, making it more engaging and effective as a product description. / Both responses follow all instructions and meet the 60-80 word count constraint. Response 2 is slightly better as its tone is more engaging and tailored specifically to gym-goers. | openai/gpt-5.6-terra: B — Both meet the 60–80-word requirement, cover all required features, and avoid the cited cliché. Response 2 is more concise, professional, and directly suited to gym-goers. Response 1 is engaging but includes more promotional phrasing and slightly less streamlined wording. / Both meet the 60–80-word requirement and include all specified features. Response 1 is more concise, professional, and fact-focused. Response 2 is engaging but adds less-supported claims (“fit perfectly,” “standard” holders) and ends with a motivational cliché, despite the instruction to avoid clichés.

Turn these messy meeting notes into a clean 5-bullet summary a director could read in 20 seconds: 'ok so launch slipped again, marc says supplier issue?? maybe 3 wks. sarah wants to push the email campaign anyway but tom thinks wait. budget - we're 8k over, mostly the packaging redo. Q: do we tell retail partners now or after new date confirmed. also NEED to hire the warehouse temp before august rush. next mtg tues.'

anthropic/claude-sonnet-5: B — Both accurate, concise, 5 bullets. Response 2 includes next meeting Tuesday detail, making it more complete while still fitting 5 bullets by combining hiring and next steps. Response 1 omits the next meeting Tuesday detail entirely. / Both accurately summarize in 5 bullets. Response 1 includes 'next meeting Tuesday' detail, slightly more complete. Response 2 omits next meeting date, minor loss of info. Both clear and concise; Response 1 marginally more useful. | google/gemini-3.1-pro-preview: B — Both responses follow all instructions and provide excellent 5-bullet summaries. Response 2 is slightly better as it manages to include the 'next meeting' detail within the 5 bullets, making it a more complete summary of the notes. / Response 1 includes all key details from the notes (including the next meeting date) while strictly adhering to the 5-bullet format without any unnecessary conversational filler. Response 2 omits the next meeting date and includes intro text. | openai/gpt-5.6-terra: B — Both provide exactly five clear, director-ready bullets and accurately capture the core issues. Response 2 is stronger because it includes the next meeting date while preserving the five-bullet format, and adds that the launch date is still TBD. Response 1 omits the Tuesday meeting detail. / Both provide exactly five accurate, director-ready bullets. Response 1 is stronger because it preserves the next-meeting detail while remaining concise. Response 2 is clear and accurate but omits the Tuesday meeting and adds an unnecessary introductory line.

Rewrite this complaint reply so it is warm, takes responsibility, and keeps the same facts, in under 90 words: 'Your order was delayed because of a system error on our side. The refund has been processed and will arrive in 3-5 days. We cannot offer further compensation. Let us know if you have questions.'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both responses are warm, accurate, take responsibility, keep facts, and are under 90 words. Quality and clarity are comparable, with R2 slightly more concise but both equally effective. / Both meet facts and warmth under 90 words. Response 1 is more concise while retaining all facts; Response 2 is slightly longer and adds extra phrases, though still under limit. Response 1 slightly better balances warmth and concision. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses follow all instructions, keep the facts, and are under 90 words. Response 2 is slightly more concise and flows better as a single cohesive message, making it ideal for a busy professional. / Both responses follow all instructions and constraints perfectly. Response 2 is slightly better as it achieves a warmer, more empathetic tone while maintaining all the original facts and staying well under the word limit. | openai/gpt-5.6-terra: B — Both are accurate, warm, responsible, and under 90 words. Response 2 is more concise and preserves the original compensation position more precisely. Response 1’s “at this time” slightly softens “cannot offer,” potentially implying future availability, and adds unnecessary wording. / Both are under 90 words, warm, accountable, and preserve the required facts. Response 1 is slightly stronger because it is concise and closely mirrors the original without adding potentially overpromising language. Response 2 is also clear and empathetic but adds “always here to help,” which may imply an unrealistic availability commitment.

Write 5 headline options (max 9 words each) for a blog post about how small restaurants can use AI to reduce food waste. Mix: 2 practical, 2 curiosity-driven, 1 with a number.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet the mix requirement, word counts under 9, clear and useful. Response 2 slightly more polished formatting and titles; Response 1 slightly wordier phrasing but both correct and compliant. / Both meet the category mix and word limits, but Response 1 has punchier, more headline-like phrasing while Response 2's headlines read as plain sentences with periods, less engaging and less typical of headline style. | google/gemini-3.1-pro-preview: B — Response 1 failed the maximum word count constraint, as every single headline is exactly 10 words long. Response 2 followed all instructions perfectly, keeping all headlines at 9 words or fewer while providing the requested mix. / Response 1 followed all instructions perfectly, including the strict word count limit per headline. Response 2 failed the negative constraint, as every single headline is exactly 10 words long, exceeding the maximum limit of 9 words. | openai/gpt-5.6-terra: B — Response 1 meets the requested mix but four headlines exceed the nine-word limit and its practical options are repetitive. Response 2 provides exactly five distinct headlines, correctly grouped into two practical, two curiosity-driven, and one numbered option; every headline is within nine words and is clear and useful. / Response 1 fully meets the five-headline, mix, and nine-word maximum requirements; all headlines are clear and relevant. Response 2 has the requested mix and relevant ideas, but headlines 1, 2, 3, and 5 each contain 10 words, breaking the explicit maximum.

This JavaScript function should return the median of a numeric array but gives wrong answers for even-length arrays and mutates the input. Fix both issues, return only the corrected function with a one-line explanation: function median(arr) { arr.sort(); const mid = Math.floor(arr.length / 2); return arr[mid]; }

anthropic/claude-sonnet-5: tie — Both responses provide identical correct code with a one-line explanation, differing only in explanation placement (before vs after code). Both fix mutation and even-length median correctly. / Both responses provide identical correct fixes with a one-line explanation, satisfying the task requirements equally well. | google/gemini-3.1-pro-preview: tie — Both responses perfectly follow the instructions, fixing the mutation, the even-length logic, and the implicit numeric sort bug. Both provide exactly the corrected function and a one-line explanation. / Both responses correctly fix the mutation, numeric sorting, and even-length array issues. Both follow the constraint to provide only the function and a one-line explanation. | openai/gpt-5.6-terra: tie — Both correctly avoid mutation by copying, use numeric sorting, and average the two middle values for even-length arrays. Each includes the corrected function and a one-line explanation. Response 2 is slightly more direct in ordering, but neither has a meaningful quality advantage. / Both correctly avoid mutating the input via a copy, use numeric sorting, and return the average of the two middle values for even-length arrays. Each provides only the corrected function plus a one-line explanation. Response 2 places the explanation before the code, but this does not reduce correctness or usefulness.

Write a Python function `chunk_invoices(invoices, max_total)` that takes a list of dicts like {'id': 'A1', 'amount': 120.5} and groups them into batches where each batch's summed amount does not exceed max_total. A single invoice larger than max_total goes in its own batch. Preserve order. Include 3 test cases using assert.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly implement the function and include valid asserts (with extras). Response 1 has verbose commented reasoning cluttering code; Response 2 is cleaner and more concise while equally correct. Both handle oversized invoices correctly. / Both implementations are correct, but Response 1 is cleaner, uses plain asserts as requested (not gated behind __main__), and lacks messy in-code reasoning/comments. Response 2 works but is verbose, includes confusing inline reasoning, and provides 4 tests instead of 3, not violating strictly but less clean. | google/gemini-3.1-pro-preview: B — Both responses correctly implement the function and include the required assertions. Response 2 provides a much more elegant and concise algorithm that naturally handles oversized invoices without needing the extra conditional branches found in Response 1. / Both responses provide correct implementations and follow all instructions. Response 1 is preferred because its logic is more elegant and concise, whereas Response 2 includes messy, stream-of-consciousness comments in the test cases. | openai/gpt-5.6-terra: tie (flipped on swap) — Both functions correctly preserve order and produce the required batching for ordinary non-negative invoice amounts, including oversized invoices. Response 1 is explicit but unnecessarily verbose and includes a fourth test case despite requesting 3. Response 2 is concise and has three labeled test cases; its oversized behavior works, though less explicitly. / Both functions correctly preserve order and isolate oversized invoices. Response 1 is concise and its logic handles the stated cases; Response 2 is also correct but includes confusing discarded expected output and verbose self-correction comments. Both provide four assert statements rather than exactly three requested test cases.

Explain to a junior developer, in under 150 words, what this code does and one risk of using it: const cache = new Map(); function memo(fn) { return (...args) => { const k = JSON.stringify(args); if (!cache.has(k)) cache.set(k, fn(...args)); return cache.get(k); }; }

anthropic/claude-sonnet-5: tie — Both correctly explain memoization and identify the memory leak risk, within word limit, clear and concise. Nearly identical quality. / Both accurately explain memoization and identify the memory leak risk clearly within word limit, in accessible language. Very similar quality; Response 2 slightly more concrete example, Response 1 slightly more concise. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses accurately explain the code and identify the memory leak risk while staying well under the 150-word limit. Response 1 is slightly better tailored to a junior developer by briefly defining memoization and explaining the mechanics a bit more clearly. / Both responses accurately explain the code and identify the memory leak risk while staying under the 150-word limit. Response 1 is more concise and gets straight to the point, making it slightly better. | openai/gpt-5.6-terra: B — Both accurately explain memoization and identify the unbounded shared cache as a valid memory-growth risk, within 150 words. Response 2 is more concise and directly structured for a busy junior developer. Response 1 is slightly less precise in calling the JSON key “unique,” since serialization can collide or fail for some values. / Both accurately explain memoization and identify unbounded shared-cache memory growth. Response 1 is more concise and precisely describes the key as JSON-stringified arguments. Response 2 is useful but inaccurately implies JSON serialization always creates a unique key; some distinct argument values can serialize identically.

Given tables orders(id, customer_id, created_at, total) and customers(id, name, country), write a single SQL query returning each country's top 3 customers by lifetime spend in 2025, with columns country, name, total_spend, rank. Use a window function. Standard PostgreSQL.

anthropic/claude-sonnet-5: tie — Both queries are functionally identical, correct, use window function as required, and return the specified columns. Minor difference in date literal casting is negligible in PostgreSQL since string literals are implicitly cast. / Both queries are functionally identical, correct, use window function as required, and meet the format. Response 2 uses explicit TIMESTAMP casting, a minor stylistic improvement but not significant. Both equally clear and useful. | google/gemini-3.1-pro-preview: tie — Both responses provide accurate, well-structured PostgreSQL queries that perfectly meet all requirements, including the window function and specific column names. They are functionally identical, with minor differences in date literal formatting. / Both responses provide accurate, well-structured PostgreSQL queries that perfectly meet all requirements, including the window function and specific column names. The only difference is minor formatting and the explicit TIMESTAMP keyword in Response 2, making them equally excellent. | openai/gpt-5.6-terra: tie — Both queries correctly aggregate 2025 order totals per customer, rank customers within each country using a window function, and return ranks 1–3. Both are valid PostgreSQL and satisfy the requested output and single-query requirement. Response 1 uses explicit timestamp literals; Response 2's date strings are also valid via PostgreSQL coercion. / Both queries correctly aggregate 2025 order totals per customer, rank customers within each country using a window function, filter to ranks 1–3, and return the required columns. Both are valid PostgreSQL and satisfy the single-query requirement. Response 2 uses explicitly typed timestamp literals, but this does not materially improve correctness here.

Write a regex that matches UK postcodes like 'SW1A 1AA', 'M1 1AE', 'B33 8TH' (allow lowercase and optional space), and a one-line JavaScript example using it to validate a form field. Briefly note one edge case your regex does NOT handle.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both provide correct regex and JS, meeting instructions. Response 1's edge case note is more detailed (mentions letter restrictions and GIR 0AA), while Response 2 only mentions GIR 0AA. Both clear and concise. / Both provide correct regex, valid JS one-liner, and a valid edge case. Response 2 uses case-insensitive flag cleanly and gives slightly more detail on edge cases. Both are concise and meet all instructions equally well. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses perfectly follow all instructions, providing accurate regexes, correct one-line JavaScript examples, and relevant edge cases. They are both concise and highly useful. / Both responses follow all instructions perfectly. Response 2 is slightly better as it uses the case-insensitive flag 'i' which makes the regex cleaner, and provides a more practical edge case explanation regarding invalid letter combinations. | openai/gpt-5.6-terra: tie (flipped on swap) — Both regexes correctly match the requested common postcode formats, permit lowercase, and allow an optional space. Response 1 is slightly more useful because its case-insensitive flag is concise and its DOM-field example is immediately usable. Both accurately note unsupported special/nonstandard cases. / Both correctly match standard postcode structures, case-insensitively, and provide one-line validation plus an excluded edge case. Response 1 is slightly more precise about allowing an optional literal space. Response 2 uses \s?, which also permits tabs and other whitespace, exceeding the stated optional-space requirement.

A bakery sells loaves at £3.20. Ingredients cost £1.10/loaf, labour £0.90/loaf, fixed costs £480/week. They sell 450 loaves/week. A supplier offers ingredients at £0.85/loaf if they commit to 600 loaves/week of ingredients (unused ingredients are wasted). Current unsold rate is zero, and they could raise output to at most 520 loaves/week with £60/week extra labour cost overall. Should they take the deal? Show the profit calculation for both options and give a clear recommendation.

anthropic/claude-sonnet-5: tie (flipped on swap) — Response 1 correctly uses labour cost as flat £405+£60=£465 (task says extra £60 overall, not per-loaf), giving £209 profit. Response 2 incorrectly recalculates labour as £520×0.90+60=£528, misinterpreting the extra cost, yielding wrong profit £146. Response 1 is more accurate per task wording. / Response 1 correctly computes labour cost as 520×0.90+60=528, giving profit £146. Response 2 incorrectly uses original 450-loaf labour (£405+£60=£465) instead of scaling to 520 loaves, yielding an inflated and incorrect profit of £209. Both are clear and well-structured, but Response 1 is factually accurate. | google/gemini-3.1-pro-preview: A — Response 1 correctly interprets the extra labour cost as £60 overall on top of the current labour cost. Response 2 incorrectly adds £60 to the base labour cost of 520 loaves, double-counting the labour for the extra loaves. / Response 2 correctly interprets the '£60/week extra labour cost overall' as a flat £60 addition to the current labour costs, whereas Response 1 incorrectly applies the per-unit labour cost to the new volume and then adds £60 on top. Response 2 is also formatted very clearly. | openai/gpt-5.6-terra: tie (flipped on swap) — Response 1 correctly treats £60 as the stated overall extra weekly labour cost, giving £465 labour and £209 profit at 520 loaves. Response 2 incorrectly adds both £0.90 per additional loaf and £60, despite £60 being the overall extra labour cost, understating profit as £146. Both show calculations and clear recommendations. / Response 1 correctly applies the £0.90 per-loaf labour cost to all 520 loaves, plus the stated £60 weekly extra labour cost, giving £146 profit. Response 2 incorrectly treats £60 as replacing the labour cost for the additional 70 loaves, overstating profit by £63. Both give the correct recommendation.

Plan the launch week for a small online course (already recorded). Resources: one founder, a part-time VA (10h), email list of 2,000, £300 ad budget. Produce a 7-day plan, one line per day, each line naming the owner. Flag the single riskiest dependency.

anthropic/claude-sonnet-5: tie — Both provide clear one-line-per-day plans with owners and a flagged risk. Response 1's lines mix two owners per line but Day-owner labeled; Response 2 uses bullet format with bolded owners, slightly clearer. Both plausible risk flags. Roughly equal quality. / Both give clear 7-day plans with owners per line and flag one riskiest dependency, satisfying instructions. Response 1 has clearer single-owner-per-task lines; Response 2's dual ownership slightly muddies 'one owner' clarity but offers plausible risk. Both equally useful and concise. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses follow the constraints well. Response 1 is slightly better as it explicitly tags the owner for each day, making it easier to scan, and correctly identifies the 10-hour VA limit as a highly practical risk for the founder. / Response 1 provides a highly realistic and practical plan that respects the VA's 10-hour limit and the small ad budget. Response 2 assigns complex ad management to a part-time VA and includes a live webinar, which is less practical. Both followed all formatting constraints. | openai/gpt-5.6-terra: tie (flipped on swap) — Response 1 is a practical launch sequence with clear daily ownership and correctly identifies the constrained VA capacity as the most controllable operational risk. Response 2 is clear and allocates the ad budget, but spends early launch-week days on preparation and assigns the founder ad optimization, reducing launch focus and operational fit. / Response 1 is more practical for limited resources, includes essential launch setup before promotion, allocates the ad budget clearly, and identifies the most revenue-critical dependency. Response 2 has a workable sequence but assumes launch infrastructure is ready and adds a webinar that may overextend the founder and VA.

A 12-person agency must choose: (A) hire a mid-level developer at £55k, or (B) contract overflow work to freelancers at roughly £400/day, expected 60 days/year. Give a recommendation in under 200 words covering cost, flexibility, quality risk, and one non-obvious factor.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correct, cover all points, under 200 words. Response 2 gives more nuanced recommendation with conditional triggers and a clearer non-obvious factor (management attention), adding more practical value while staying concise. / Both correctly recommend B, cover cost/flexibility/quality/non-obvious factor, stay under 200 words, and are clear and useful. Nearly equivalent quality with minor stylistic differences; both accurate and well-structured. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses perfectly follow all instructions, including the word count constraint. They accurately address cost, flexibility, quality risk, and provide excellent non-obvious factors. Both are highly useful, clear, and concise for a busy professional. / Both responses perfectly follow the constraints, staying under 200 words and covering all required points. Response 1's non-obvious factor (management overhead) is slightly more insightful for a small agency than Response 2's (knowledge fragmentation), which borders on a standard quality risk. | openai/gpt-5.6-terra: B — Both are under 200 words and cover all requested factors accurately. Response 2 is stronger because it more clearly qualifies the recommendation, acknowledges the employee’s quality/ownership advantages, and gives a practical review threshold. Response 1 is also sound, though its cost-premium framing is slightly more assumption-dependent. / Both are under 200 words and cover all required areas. Response 1 is more balanced, clearly caveats the recommendation, and offers a genuinely non-obvious management-attention factor. Response 2 is strong and practical, but “scale instantly” is overstated and its savings/reinvestment conclusion is more assertive than necessary.

A tank holds 2,400 litres. Pump A fills at 40 L/min. Pump B drains at 25 L/min. A runs from 09:00. B accidentally switches on at 09:20. At 10:00 B is switched off. At what time is the tank full? Show your working.

anthropic/claude-sonnet-5: tie — Both responses correctly compute the same answer, 10:25, with clear, accurate step-by-step working. Equally clear, concise, and useful. / Both responses are correct, follow instructions, show clear working, and arrive at the same accurate answer of 10:25. Response 2 is slightly more readable with formatting, but both are equally correct and useful. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses are perfectly accurate and show clear step-by-step working. Response 1 is slightly better for a non-technical professional as it avoids LaTeX formatting delimiters, making it marginally easier to read. / Both responses correctly calculate the final time as 10:25, showing clear, step-by-step working. Response 1 is slightly more concise, while Response 2 uses bullet points for readability. Both perfectly answer the prompt. | openai/gpt-5.6-terra: tie — Both responses correctly calculate 800 L by 09:20, 600 L added during the 40 minutes of combined operation, 1,400 L at 10:00, and 25 more minutes at 40 L/min. Both show working, follow all instructions, and are clear and useful. / Both responses correctly calculate 800 L by 09:20, 600 L net during 09:20–10:00, then 25 minutes to fill the remaining 1,000 L, giving 10:25. Both show working, follow all task requirements, and are clear. Response 1 is slightly more concise; Response 2 is similarly useful and readable.

Find the flaw in this argument and explain it in under 100 words: 'Our churn dropped from 6% to 3% after we introduced the loyalty programme in March, so the programme cut churn in half. We should double the loyalty budget to cut churn to 1.5%.'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly identify correlation/causation and linear extrapolation flaws, are under 100 words, clear and useful. Very similar quality; Response 2 slightly more concise but both equally accurate and thorough. / Both correctly identify correlation/causation and diminishing returns flaws within the word limit. Response 2 is slightly more detailed and structured, aiding clarity for a non-technical reader, while Response 1 is more concise but slightly less thorough. | google/gemini-3.1-pro-preview: A — Both responses correctly identify the two main logical flaws (correlation vs. causation and linear extrapolation) and stay well under the 100-word limit. Response 1 is slightly better formatted with clear bullet points, making it more readable for a busy professional. / Both responses correctly identify the flaws and stay under the word limit. Response 2's use of bullet points makes it slightly easier to read for a busy professional. | openai/gpt-5.6-terra: tie (flipped on swap) — Both correctly identify unsupported causation and unjustified linear scaling, and both are under 100 words. Response 2 is more concise, directly states the inferential errors, and adds the useful point that churn has a floor. Response 1 is also clear but slightly more verbose. / Both accurately identify the two core flaws: post hoc correlation does not establish causation, and the assumed proportional effect of additional spending is unsupported due to diminishing returns and a practical churn floor. Both are clear, concise, actionable, and under 100 words.

Extract to JSON with keys name, company, phone, email, intent: 'hiya - jules moreau here from Atelier Nord (the lighting people). best number is 07700 900 457, or jules@ateliernord.fr. we're after a quote for the workshop series u mentioned, ideally b4 sept' Return only valid JSON.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly extract JSON with required keys. Response 1 uses properly capitalized name and intent, slightly more polished. Response 2 keeps lowercase from original, less clean but still accurate. Both valid JSON, no constraint violations. / Both correctly extract data into valid JSON with required keys; only minor casing differences in name/intent, neither breaking instructions. Equally clear, concise, and accurate. | google/gemini-3.1-pro-preview: A — Both responses correctly extract the requested information into valid JSON. Response 1 is slightly better as it capitalizes the name and formats the intent more professionally. / Both responses correctly extract the requested information into valid JSON. Response 2 is slightly better as it normalizes the capitalization of the name and expands the abbreviations in the intent, making it more professional. | openai/gpt-5.6-terra: tie (flipped on swap) — Both accurately extract all requested fields and return valid JSON only. Response 1 is slightly stronger because it standardizes capitalization and expands “sept” to “September,” making the result more polished and immediately usable. Response 2 remains accurate but retains informal casing and abbreviation. / Both return valid JSON with all required keys and accurately extract the contact details and intent. Response 1 preserves “ideally,” while Response 2 improves capitalization and expands “Sept” for clarity; neither meaningfully harms accuracy or usefulness.

Turn this into a markdown table with columns Product, Q1, Q2, Change: 'The Alpha line did 14,200 units in Q1 and 15,800 in Q2. Beta slipped from 9,100 to 7,400. The new Gamma launched mid-Q2 with 3,300 units (no Q1 sales). Delta held flat at 5,000 both quarters.' Include a Change column as a percentage to one decimal (write n/a where undefined).

anthropic/claude-sonnet-5: tie — Both responses are identical in content and correctness, only differing in table formatting spacing which doesn't affect rendering. Both satisfy all constraints accurately. / Both responses are identical in content and correctly calculate percentage changes with n/a for Gamma; only formatting of table delimiters differs, which is inconsequential. | google/gemini-3.1-pro-preview: tie — Both responses perfectly follow all instructions, accurately calculating the percentage changes to one decimal place and formatting the output as a Markdown table with the requested columns. / Both responses perfectly follow all instructions, accurately calculating the percentages to one decimal place and formatting the output as a markdown table with the requested columns. | openai/gpt-5.6-terra: tie — Both responses accurately extract all quarterly unit figures, calculate percentage changes correctly to one decimal, use n/a for undefined Gamma growth, and provide the requested markdown table with the exact columns. Response 2 has slightly more conventional separator formatting, but both are valid and equally useful. / Both responses accurately extract all quarterly values, calculate percentage changes correctly to one decimal, use n/a for Gamma’s undefined change, and provide the requested markdown table columns. Both are clear and concise.

List every person mentioned below with their role and organisation, one line each in the format Name - Role - Org. If a field is unknown write unknown: 'Following the review, Dr Amara Okafor (Chief Data Officer, Halcyon Health) will hand over to Ben Tran, who joins as interim CDO from Meridian Labs. Legal sign-off sits with Priya Shah at Halcyon; the external audit remains with T. Lindqvist.'

anthropic/claude-sonnet-5: tie — Both responses are identical, accurate, and follow the format correctly. / Both responses are identical, correctly listing all four individuals with role and org, using 'unknown' where appropriate, matching the required format exactly. | google/gemini-3.1-pro-preview: tie — Both responses are identical and perfectly follow all instructions, accurately extracting the names, roles, and organizations in the requested format. / Both responses are identical and perfectly follow all instructions, extracting the correct names, roles, and organizations in the requested format. | openai/gpt-5.6-terra: tie — Both responses correctly identify all four people, infer roles and organisations appropriately, use unknown where organisation is absent, and follow the required one-line Name - Role - Org format exactly. / Both responses correctly list all four people, accurately infer roles and organisations, use unknown appropriately for T. Lindqvist’s organisation, and follow the required one-line format exactly.

Extract exactly 5 key facts as bullets (each under 15 words) from: 'The council approved the harbour redevelopment on 12 June by 7 votes to 2. Phase one, costing £4.6m, begins in October and includes the fish market roof. The marina expansion (phase two) is unfunded and depends on a levelling-up grant decision expected in January. Local traders get first refusal on the 14 new retail units. Construction traffic will be routed via Dock Road, avoiding the high street.'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both extract accurate, concise 5-bullet facts under 15 words each, following instructions well. Response 2 is slightly more concise, but both are equally correct and useful. / Both accurate and follow instructions with exactly 5 bullets under 15 words. Response 1 is slightly more concise; Response 2's second bullet is borderline near 15 words. Both equally useful and clear, but Response 1 edges out on concision. | google/gemini-3.1-pro-preview: tie — Both responses perfectly followed all instructions, extracting exactly 5 key facts as bullets with each bullet under the 15-word limit. Both are accurate, clear, and highly useful. / Both responses perfectly follow all instructions, extracting exactly 5 key facts as bullets with each bullet well under the 15-word limit. They are both highly accurate, clear, and useful. | openai/gpt-5.6-terra: tie (flipped on swap) — Both provide exactly five bullets, each under 15 words, and accurately capture the key facts. Response 1 is slightly stronger because it explicitly retains that the January item is a grant decision; Response 2’s wording could imply the grant itself occurs in January. / Both provide exactly five bullet points, each under 15 words, and accurately capture all five key facts. Both are concise, clear, and suitable for a busy professional.

Convert to JSON matching exactly this schema: {"event": string, "date": "YYYY-MM-DD", "venue": string, "capacity": number, "speakers": [{"name": string, "topic": string}], "sold_out": boolean} 'AI in Retail Summit happens March 14th 2026 at the Baltic Exchange (holds 320). Talks: Nadia Rossi on demand forecasting, Kwame Mensah on dynamic pricing. Tickets still available.' Return only the JSON.

anthropic/claude-sonnet-5: tie — Both responses correctly produce identical valid JSON matching the schema exactly, differing only in formatting whitespace. / Both correctly match the schema with accurate data; formatting differs (compact vs pretty) but both are valid JSON with no extra text, satisfying instructions equally. | google/gemini-3.1-pro-preview: tie — Both responses follow all instructions perfectly, extracting the correct information and formatting it exactly according to the requested JSON schema. They both return only the JSON as requested. / Both responses correctly extract the information and format it exactly according to the requested JSON schema. Both follow the negative constraint to return only the JSON. | openai/gpt-5.6-terra: tie — Both responses are valid JSON and exactly match the required schema. They accurately convert the date, capacity, speakers and topics, and infer sold_out as false from tickets still being available. Response 1 is formatted for readability; Response 2 is more compact. Neither breaks any explicit constraint. / Both responses are valid JSON and exactly match the required schema and source details. Dates, venue, capacity, speakers, topics, and sold-out status are all correct. Response 1 is more compact, but Response 2 remains clear and compliant.

Describe how photosynthesis works in exactly 3 sentences. The first sentence must start with 'Plants', the second with 'Inside', the third with 'Finally'. No sentence may exceed 20 words.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet constraints: 3 sentences, correct starters, under 20 words each. Response 1 is slightly more accurate and clear; Response 2 lacks commas after 'Plants' and 'Finally', slightly odd phrasing. / Both meet the 3-sentence structure, correct starting words, and word limits. Content is accurate and clear in both; R2 slightly more precise about glucose/water/oxygen relationship, but overall quality is comparable. | google/gemini-3.1-pro-preview: A — Both responses follow all constraints perfectly. Response 1 is slightly better written with proper punctuation (commas after introductory phrases), making it clearer and more professional. / Both responses follow all constraints perfectly. Response 2 is the winner because it uses proper punctuation, making it clearer and easier to read, whereas Response 1 is missing commas. | openai/gpt-5.6-terra: B — Both meet the exact three-sentence structure, required starting words, and 20-word limit. Response 2 is more accurate and clearly explains chlorophyll’s role, glucose formation, and oxygen release. Response 1 is understandable but inaccurately implies light energy converts water directly into glucose, omitting carbon dioxide’s role in glucose production. / Both meet the three-sentence, opening-word, and word-limit constraints. Response 1 is clearer and more accurate overall, though it compresses glucose formation into light-driven reactions. Response 2 inaccurately suggests light energy converts water directly into glucose; carbon dioxide supplies the carbon for glucose.

Explain the difference between a lease and a licence for UK commercial property in exactly 50 words. Count carefully - exactly 50.

anthropic/claude-sonnet-5: tie — Both responses are exactly 50 words, accurate, clear, and well-structured explanations of lease vs licence distinctions. Content quality and concision are comparable, with no constraint violations in either. / Both responses are exactly 50 words, accurate, clear, and cover the key distinctions between leases and licences in UK commercial property law. Quality and usefulness are essentially equivalent. | google/gemini-3.1-pro-preview: B — Both responses perfectly meet the strict 50-word constraint and accurately explain the legal distinction. Response 2 is slightly better for a non-technical professional as it explicitly mentions UK commercial property and concludes with a practical summary of security versus flexibility. / Both responses perfectly meet the strict 50-word constraint and accurately explain the legal distinction. Response 1 is slightly better as it explicitly mentions 'UK commercial property' and concludes with a practical takeaway (security vs flexibility) that is highly useful for a non-technical professional. | openai/gpt-5.6-terra: B — Both are exactly 50 words and accurately state the core distinction: exclusive possession and an estate for leases versus personal permission for licences. Response 2 is more useful and balanced, explaining practical consequences. Response 1 overstates that a licensor retains control of the entire premises; Response 2 slightly simplifies revocability. / Both meet the exact 50-word requirement and clearly explain the core distinction. Response 1 more accurately identifies a lease’s defined term and proprietary nature. Its statement that licences are easily revocable is somewhat broad. Response 2 wrongly implies leases must have fixed terms and overstates the licensor’s retained control.

Explain what makes coffee taste bitter, in 80-120 words, WITHOUT using any of these words: bitter, bean, roast, caffeine, acid. Do not use hyphenated or partial variants of them either.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet constraints and word count, avoid banned words. Response 1 gives clearer explanation of bitterness causes (heat, chemical breakdown), while Response 2 is slightly less focused on bitterness itself, more on extraction technique. / Both meet word count and avoid banned words/variants. Response 1 is clearer, more concise, and directly ties causes to practical brewing tips, while Response 2 is slightly less focused and repetitive. | google/gemini-3.1-pro-preview: A — Both responses follow all constraints, including word count and forbidden words. Response 1 provides a more direct and comprehensive explanation of the chemical and physical causes of the harsh flavor, whereas Response 2 spends too much time on how to prevent it. / Both responses follow all negative constraints and the word count limit. Response 2 provides a slightly more comprehensive explanation by addressing both the heating process of the seeds and the brewing extraction, making it more accurate and useful. | openai/gpt-5.6-terra: A — Both meet the 80–120-word requirement and avoid all prohibited terms and variants. Response 1 more fully explains heat-driven chemical changes and excessive extraction. Response 2 is practical and clear but somewhat oversimplifies the causes by emphasizing over-extraction and making a less precise claim about heating generating substances. / Both meet the 80–120-word limit and avoid all prohibited terms and variants. Response 2 more accurately covers both heating-related chemical changes and excessive extraction. Response 1 is clear and practical but overemphasizes over-extraction and makes less precise claims about polyphenols and production heating.

Write a LinkedIn post about hiring your first apprentice. Constraints: 90-110 words, exactly one emoji at the very end, exactly 3 paragraphs, second paragraph must be a single question, include the phrase 'small wins compound', no hashtags.

anthropic/claude-sonnet-5: A — Both meet word count (102, 104), paragraph structure, single emoji at end, required phrase, no hashtags. R1 reads more naturally and engagingly, while R2 feels slightly more generic/wordy. / Both meet word count, structure, phrase, and emoji constraints. Response 2 reads more naturally and engagingly, while Response 1 is slightly repetitive and stiff. | google/gemini-3.1-pro-preview: A — Response 1 followed all constraints perfectly, including word count, paragraph count, and specific phrases. Response 2 violated the paragraph constraint by placing the emoji on a new line, resulting in 4 paragraphs instead of exactly 3. / Response 2 followed all constraints perfectly, including word count, paragraph count, and formatting. Response 1 placed the emoji on a new line, creating a fourth paragraph, which violates the exactly 3 paragraphs constraint. | openai/gpt-5.6-terra: A — Both satisfy every explicit constraint: word count, three paragraphs, single-question second paragraph, required phrase, no hashtags, and one final emoji. Response 1 is more natural, concise, and engaging for LinkedIn. Response 2 is accurate and compliant but more generic and wordy. / Response 1 is 104 words and otherwise meets the content requirements, but the standalone emoji creates a fourth paragraph. Response 2 is 101 words, has exactly three paragraphs, uses the required phrase, contains no hashtags, and places exactly one emoji at the end. It is clearer and fully compliant.

Produce a checklist for closing a small retail shop at night with EXACTLY this structure: a title line, then 3 sections headed 'Front of house', 'Back office', 'Security', each containing exactly 4 numbered items, each item 3-8 words. Nothing else before or after.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both follow structure closely with correct sections and item counts, word counts within range. Response 1 has title on its own line with blank line separation, cleaner formatting. Response 2 lacks blank lines between title and sections, slightly less clean but still compliant. / Both follow structure with 4 items/section, 3-8 words each. Response 1 has no blank lines, matching stricter 'nothing else' interpretation; Response 2 adds blank lines between title and sections, slightly deviating from exact structure. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses follow all constraints perfectly, including section headers, item counts, and word counts per item. Response 2 is slightly better formatted as a continuous list without unnecessary blank lines, making it slightly more concise. / Both responses follow all instructions perfectly, including the exact structure, section headings, item counts, and word count constraints per item. Both provide useful and clear checklists for closing a retail shop. | openai/gpt-5.6-terra: B — Both meet the exact structure and item word-count requirements. Response 2 is more practical and concise, especially its camera verification and cash-security steps. Response 1 includes “Set up outdoor security cameras,” which is not typically a nightly closing task, and activates the alarm before checking doors and windows. / Both meet the required headings, section/item counts, and 3–8-word item limits. Response 1 is complete, practical, and concise. Response 2 is also clear, but “Set up outdoor security cameras” is less appropriate as a nightly closing task than verifying cameras are operating.

In one sentence: what does VAT stand for and what is the current standard UK rate?

anthropic/claude-sonnet-5: tie — Both responses are identical, correct, concise, one-sentence answers meeting the task requirements exactly. / Both responses are identical, correct, concise, and answer in one sentence as instructed. | google/gemini-3.1-pro-preview: tie — Both responses are identical, perfectly accurate, and follow all instructions, including the one-sentence constraint. / Both responses are identical, factually correct, and perfectly follow the constraint to provide the answer in exactly one sentence. | openai/gpt-5.6-terra: tie — Both responses are factually correct, answer in one sentence, and clearly state that VAT means Value Added Tax and that the standard UK rate is 20%. / Both responses are factually correct, satisfy the one-sentence instruction, and clearly provide the requested definition and current standard UK VAT rate.

Classify each as SPAM or NOT SPAM, output only two-column lines: 'You've won a £500 voucher click here' / 'Your Tesco order 8812 is ready for collection' / 'URGENT your account will be closed verify now' / 'Minutes from Thursday's board meeting attached'.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly classify all four items. Response 2 keeps original quotes and slash format matching the task's style more closely, while Response 1 uses pipe instead of slash - minor formatting deviation, but both are equally correct and clear. / Both correctly classify all four items. Response 1 uses '/' as requested delimiter matching task format exactly, while Response 2 uses '|' instead, slightly deviating from the specified two-column format style shown in the task. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses correctly classify the messages and follow the formatting constraint. Response 1 is slightly better as it uses a standard pipe separator for columns and removes the unnecessary quotation marks. / Both responses correctly classify the messages and follow the constraint to output only two-column lines. Response 1 retains the quotes from the prompt, while Response 2 removes them, but both formats are perfectly acceptable. | openai/gpt-5.6-terra: tie — Both correctly classify all four messages and provide only two-column line outputs. Response 1 uses a pipe separator; Response 2 uses slashes and retains quotation marks. Neither format choice conflicts with an explicit required separator or quoting requirement. / Both correctly classify all four messages and provide only two-column output lines. Response 1 uses slash delimiters and Response 2 uses pipe delimiters; both are clear, concise, and satisfy the requested format.

Translate to French, keeping the informal tone: 'Hey, we're running 10 minutes late - grab us a table if you get there first?'

anthropic/claude-sonnet-5: tie — Both provide accurate, informal French translations with alternatives. Response 1's phrasing is slightly more natural; Response 2's 't'arrives' is a bit informal but fine. Both similar quality and clarity. / Both provide accurate, informal French translations with alternatives. Response 1 is slightly more concise; Response 2 offers more explanation, equally valid. No constraints violated in either. | google/gemini-3.1-pro-preview: B — Both provide excellent, accurate translations that capture the informal tone. Response 2's use of 't'arrives en premier' feels slightly more natural and closer to the colloquial feel of the original English prompt. / Both models provide accurate and informal translations. Response 1 captures the casual tone slightly better with the contraction 't'arrives' and is more concise. | openai/gpt-5.6-terra: B — Both convey the intended informal message. Response 2 is more concise and its main phrasing is natural enough, though “prends-nous une table” is slightly less idiomatic than “garde/réserve-nous une table.” Response 1 adds unnecessary explanation and alternatives; “prendre une table” is also somewhat awkward. / Both provide accurate, informal French translations. Response 1 is more direct and immediately usable, though its alternative is unnecessary. Response 2 is natural and correctly conveys the meaning, but the English framing and multiple options are less concise; its first version also softens the original imperative into a question.

Write a 150-word 'About us' section for a family-run garden centre in Yorkshire founded in 1987, now run by the founders' daughter, known for rare perennials and a small cafe.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet word count and content requirements. Response 2 is slightly more vivid, detailed, and polished, with clear third-person narrative matching the daughter's role, while Response 1's first-person voice is less consistent with a general 'About Us' tone. / Both meet content requirements well; word counts close to 150. Response 2 uses first-person voice, adding warmth; Response 1 slightly more polished. Both similarly clear, concise, and accurate. | google/gemini-3.1-pro-preview: tie — Both responses fail the exact 150-word constraint (Response 1 is 148 words, Response 2 is 154 words). However, both successfully include all the requested details in a clear, engaging, and professional tone suitable for a garden centre's 'About Us' section. / Both responses fail the strict 150-word count constraint. Response 2 is slightly better as it uses a first-person perspective ('my parents', 'I am proud'), which feels much more natural and engaging for a family-run business's 'About us' page compared to Response 1's third-person approach. | openai/gpt-5.6-terra: tie — Both are accurate, polished, and include all requested details. Response 1 is 148 words; Response 2 is 151 words without its heading (153 with it), so both miss the exact 150-word requirement. Response 1 is closer to the target and has a more personal family-successor voice. / Response 1 is 151 words including its heading (149 without it), while Response 2 is 148 words, so both miss the required 150 words. Both accurately cover the family history, daughter, rare perennials and café. Response 1 is more specific and polished; Response 2’s first-person framing is less clearly appropriate for a general About Us section.

Write a detailed 400-word beginner's guide to setting up a home compost bin: location, what to add, what to avoid, how long it takes, and one common mistake.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both responses cover all required elements thoroughly and accurately with clear structure. Both are close to 400 words (likely slightly over/under but not strictly verifiable as violation). Response 2 is slightly more detailed and practical, but both are equally clear and useful. / Both cover all required elements accurately and clearly. Response 1 is slightly more detailed, includes word count confirming ~412 words meeting 400-word target, and adds more nuance (moisture, ratios). Response 2 is solid but slightly less detailed on location and moisture guidance. | google/gemini-3.1-pro-preview: tie — Both responses fail the exact 400-word constraint (Response 1 is ~388 words, Response 2 is ~416 words). However, Response 2 provides slightly more practical details (like the 3x3 foot area, moisture level comparison) making it more useful for a beginner. / Both responses provide excellent, clear, and accurate guides covering all required points. However, both fail the strict 400-word count constraint (Response 1 is ~410 words, Response 2 is ~420 words). They tie on quality. | openai/gpt-5.6-terra: tie — Both cover every requested topic accurately and clearly, but neither is exactly 400 words: Response 1 is 419 words and Response 2 is 412 words. Response 2 is more practical and structured, while Response 1 is slightly more concise and avoids a few overbroad cautions in Response 2. / Both fail the explicit 400-word requirement: Response 1 states 412 words; Response 2 is about 419 words. Response 1 is more detailed and fully addresses every requested topic with practical moisture, aeration, and troubleshooting advice. Response 2 is clear and accurate but less comprehensive, especially on bin management and acceptable materials.