Probably Gemini 3.5 Flash

Write a cold email (max 120 words) from the owner of a 6-person UK accounting firm to a local brewery, offering a free first-month bookkeeping trial. Friendly but professional, one clear call to action, no buzzwords.

anthropic/claude-sonnet-5: A — Both meet word limit, tone, and CTA requirements. Response 1 is more personable and specific (team size, favourite beer), fitting the 6-person firm detail. Response 2 is solid but slightly less warm and omits firm size detail. / Both meet constraints and word limit, friendly/professional tone, one CTA, no buzzwords. Response 2 has slightly more specific personalization and clearer CTA (specific day/time), making it marginally more effective for a busy reader. | openai/gpt-5.6-terra: tie (flipped on swap) — Response 1 clearly identifies a six-person local firm, offers a free first month, uses brewery-specific context, and has one direct call to action. Response 2 is concise and professional but omits the required six-person detail and does not establish the firm as UK-based. / Response 1 is concise, friendly, and has one clear CTA, but does not state the firm has six people or is UK-based. Response 2 includes the six-person detail and a clear CTA, but exceeds 120 words (about 125), omits explicit UK context, and includes an awkward beer placeholder. | x-ai/grok-4.5: A — Both under 120 words, no buzzwords, clear CTA. R1 better meets all specs (mentions 6-person firm) and is more personalized/useful. / Both under 120 words, professional, one CTA, no buzzwords. Resp2 correctly notes 6-person firm and is more tailored/useful; Resp1 omits size detail.

Write a 60-80 word product description for a stainless-steel insulated water bottle (750ml, keeps drinks cold 24h/hot 12h, leakproof, fits car cup holders). Target audience: gym-goers. Avoid cliches like 'stay hydrated in style'.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both are factually accurate, avoid cliches, and target gym-goers well. R1 meets the 60-80 word range (67 words) while R2 falls short at 56 words, violating the explicit word count constraint. R1 is slightly more detailed and complete. / Both meet word count and avoid clichés. Response 1 (63 words) is tighter and more vivid; Response 2 (72 words) uses 'ultimate companion,' a mild cliché, slightly weaker. Both accurate and on-brief; Response 1 edges out on concision and freshness. | openai/gpt-5.6-terra: A — Response 1 is 66 words, covers every specification, targets gym-goers, and avoids the prohibited cliché. Response 2 is engaging and covers the core features, but it is only 55 words, below the required 60–80 range. Its unsupported “sweat-free” claim is also less precise. / Response 1 is accurate, gym-focused, and clear, but contains 56 words, below the required 60–80 range. Response 2 meets the word count at 68 words and covers all features clearly. Both use effective, non-technical language; Response 2 is slightly more promotional and includes a mild overclaim (“guarantees zero spills”). | x-ai/grok-4.5: A — Resp1 hits 68 words, covers all specs accurately for gym audience without cliches. Resp2 is only 57 words (under limit), otherwise solid but incomplete on length constraint. / Both cover specs accurately for gym audience without cliches and are clear/useful. R1 has 57 words; R2 has 68 and slightly stronger flow.

Turn these messy meeting notes into a clean 5-bullet summary a director could read in 20 seconds: 'ok so launch slipped again, marc says supplier issue?? maybe 3 wks. sarah wants to push the email campaign anyway but tom thinks wait. budget - we're 8k over, mostly the packaging redo. Q: do we tell retail partners now or after new date confirmed. also NEED to hire the warehouse temp before august rush. next mtg tues.'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both accurate and concise. Response 2 includes 'next meeting Tuesday' detail, omitted in Response 1, making it slightly more complete while staying within 5 bullets. / Both accurately capture all points in 5 clean bullets, concise and director-ready. Response 1 includes next meeting note, slightly more complete; Response 2 slightly more polished wording. Essentially equal quality. | openai/gpt-5.6-terra: B — Both provide exactly five clear, accurate bullets. Response 1 is polished but omits the next meeting date and presents the supplier cause more definitively than the notes. Response 2 captures every key point, including ownership of the campaign disagreement and Tuesday’s meeting, while remaining more concise. / Both are accurate, concise, and use exactly five bullets. Response 1 captures every material note, including the Tuesday meeting, while Response 2 omits it. Response 2 is slightly more polished in phrasing, but Response 1 is more complete and equally director-friendly. | x-ai/grok-4.5: tie (flipped on swap) — Both deliver accurate 5-bullet summaries covering all key points clearly and concisely for a director. No constraints breached. / Both accurate 5-bullet summaries. R1 includes all key points incl. next mtg; R2 omits it. Equal clarity/usefulness.

Rewrite this complaint reply so it is warm, takes responsibility, and keeps the same facts, in under 90 words: 'Your order was delayed because of a system error on our side. The refund has been processed and will arrive in 3-5 days. We cannot offer further compensation. Let us know if you have questions.'

anthropic/claude-sonnet-5: tie — Both accurately preserve facts, are warm, take responsibility, and are under 90 words. Response 2 is slightly more concise; Response 1 slightly warmer. Essentially equivalent quality. / Both responses accurately preserve facts, take responsibility, stay under 90 words, and are warm and clear. R2 is slightly more empathetic with added phrasing, but both are equally effective and concise. | openai/gpt-5.6-terra: B — Both are warm, clear, under 90 words, and retain the key facts. Response 1 more explicitly accepts responsibility but changes “3–5 days” to “3–5 business days,” adding an unsupported detail. Response 2 preserves the timeline exactly and is more concise, though its ownership language is slightly less explicit. / Response 1 is warm, accountable, accurate, and concise while preserving all facts. Response 2 is empathetic and clear, but changes “3–5 days” to “3–5 business days,” which does not keep the same facts exactly. Response 1 is therefore more compliant and reliable. | x-ai/grok-4.5: tie (flipped on swap) — Both warm, responsible, fact-preserving, under 90 words, clear and useful. Minor phrasing differences only; equally strong. / Both under 90 words, warm, responsible, clear. R1 keeps exact facts (3-5 days). R2 adds 'business' altering fact slightly and is less concise.

Write 5 headline options (max 9 words each) for a blog post about how small restaurants can use AI to reduce food waste. Mix: 2 practical, 2 curiosity-driven, 1 with a number.

anthropic/claude-sonnet-5: tie — Both meet the 5-headline, word-limit, and category-mix requirements accurately. Response 2 headlines feel slightly more polished and varied in structure, while Response 1's are functional but repetitive in phrasing. Both are clear and useful; quality is close. / Both meet the mix and word-count constraints, are clear, accurate, and useful. Response 2 explicitly labels word counts, aiding verification; Response 1 is slightly more concise formatting. Quality is essentially equivalent. | openai/gpt-5.6-terra: tie (flipped on swap) — Response 1 better ties every headline set to small restaurants and AI, with the requested mix and valid lengths. Its claim that headlines are “under” nine words is inaccurate because they are exactly nine. Response 2 is concise and compliant, but several headlines omit either AI or the small-restaurant focus. / Both provide exactly five headlines, meet the 9-word maximum, and follow the requested 2 practical/2 curiosity-driven/1 numbered mix. Response 1 is slightly stronger because every headline clearly connects AI, food waste, and restaurants. Response 2’s third curiosity headline does not explicitly mention AI or food waste. | x-ai/grok-4.5: tie — Both deliver exactly 5 headlines meeting the mix and strict ≤9-word limit, with clear labeling. Equally accurate, useful, and concise for the audience. / Both deliver exactly 5 headlines under 9 words with the required 2/2/1 mix. Accurate, clear, and useful for the audience. No differences warrant a preference.

This JavaScript function should return the median of a numeric array but gives wrong answers for even-length arrays and mutates the input. Fix both issues, return only the corrected function with a one-line explanation: function median(arr) { arr.sort(); const mid = Math.floor(arr.length / 2); return arr[mid]; }

anthropic/claude-sonnet-5: tie — Both correctly fix mutation and even-length median bugs with identical logic. Response 1 includes a clearer one-line explanation as requested; Response 2 lacks an explicit explanation, slightly violating the 'one-line explanation' instruction. / Both fixes are correct and avoid mutation. Task required a one-line explanation; Response 1 omits any explanation entirely, while Response 2 provides one, though longer than strictly one line. Response 2 better fulfills instructions overall. | openai/gpt-5.6-terra: A — Both functions correctly avoid mutation, sort numerically, and average the two middle values for even-length arrays. Response 1 also includes the required one-line explanation. Response 2 omits the required explanation, though its code is otherwise correct and concise. / Both functions correctly avoid mutation, sort numerically, and average the two middle values for even lengths. Response 1 omits the required one-line explanation. Response 2 fully satisfies the requested output and clearly explains the fixes. | x-ai/grok-4.5: tie (flipped on swap) — Both correctly fix sort/mutation/even median. Both violate 'only...with a one-line explanation' (R1 multi-line; R2 none). R1 more complete/useful. / Both functions correctly fix sort mutation and even-length median. Resp1 omits required one-line explanation. Resp2 supplies correct function plus proper one-line explanation.

Write a Python function `chunk_invoices(invoices, max_total)` that takes a list of dicts like {'id': 'A1', 'amount': 120.5} and groups them into batches where each batch's summed amount does not exceed max_total. A single invoice larger than max_total goes in its own batch. Preserve order. Include 3 test cases using assert.

anthropic/claude-sonnet-5: A — Response 1 is clean, correct, and passes its asserts. Response 2's test 1 has malformed/incorrect code (duplicate list literal with stray comments), causing the assert to fail or be syntactically broken, violating the 'include working test cases' requirement. / Response 1's test function contains messy inline reasoning comments and confusing duplicate/incorrect assert blocks, making it unclear and possibly broken. Response 2 is clean, correct, well-structured, with accurate test cases matching the algorithm's logic. | openai/gpt-5.6-terra: A — Response 1’s implementation is correct, preserves order, handles oversized invoices, and is clear. It includes four assert statements despite requesting three test cases. Response 2’s implementation is correct, but its first expected result mistakenly includes an extra invalid batch ([A1, A2]), so the asserted test fails and makes the provided code unusable as written. / Response 1's implementation is correct, but its first assert has an erroneous extra expected batch ([A1, A2]) before the corrected batches, so the tests fail. Response 2 correctly preserves order, handles oversized invoices as singleton batches, and provides clear passing tests. Its third labeled case also checks empty input. | x-ai/grok-4.5: A — Resp1 is correct, clean, runnable with valid asserts. Resp2 has broken test syntax (comments inside list literal) so tests fail; messier and less useful. / Resp1 function is mostly correct but tests are invalid Python (malformed assert list with interrupting comments) so unusable. Resp2 is clean, correct, complete with working asserts.

Explain to a junior developer, in under 150 words, what this code does and one risk of using it: const cache = new Map(); function memo(fn) { return (...args) => { const k = JSON.stringify(args); if (!cache.has(k)) cache.set(k, fn(...args)); return cache.get(k); }; }

anthropic/claude-sonnet-5: A — Both correct and concise, under 150 words. Response 1 identifies a subtle, code-specific bug (shared cache across functions) with a fix, which is more insightful. Response 2's memory leak risk is valid but more generic. Both clear and useful; Response 1 slightly more precise. / Both correctly explain memoization under 150 words. Response 2 identifies a more critical, specific bug (shared cache causing collisions between different functions) with a clear fix, while Response 1's memory-leak risk is valid but less severe/specific. | openai/gpt-5.6-terra: A — Both are accurate, concise, and under 150 words. Response 1 identifies the more immediate correctness bug: different memoized functions can collide because they share keys in one global cache. Response 2 correctly notes unbounded memory growth, but calls it a definite memory leak rather than a potential retention risk. / Both are accurate, clear, under 150 words, and identify a genuine risk. Response 1 correctly explains unbounded cache growth. Response 2 identifies the more immediate correctness bug: a global cache lets different wrapped functions collide on identical argument keys, and it provides the precise fix. | x-ai/grok-4.5: tie (flipped on swap) — Both accurately explain memoization and a valid risk (shared cache vs unbounded growth), stay under 150 words, and are clear/useful for juniors. Equal quality. / Both accurately describe memoization and valid risks under 150 words. R2 highlights the critical shared-cache correctness bug more usefully for juniors; R1's memory-leak point is secondary.

Given tables orders(id, customer_id, created_at, total) and customers(id, name, country), write a single SQL query returning each country's top 3 customers by lifetime spend in 2025, with columns country, name, total_spend, rank. Use a window function. Standard PostgreSQL.

anthropic/claude-sonnet-5: A — Both queries correctly implement the task with window functions, CTEs, and top-3 filtering. Response 1 uses range comparison for date filtering (more index-friendly) versus EXTRACT which is less performant. Response 1 also provides clear explanation for a non-technical audience. / Both queries are correct and use window functions properly. Response 2 uses more index-friendly date range filtering and provides helpful explanation, slightly more useful for verification, though more verbose than needed. | openai/gpt-5.6-terra: B — Both provide valid PostgreSQL queries using window functions, aggregate 2025 spend per customer, partition rankings by country, and return the requested columns. Response 2 is more concise and separates aggregation from ranking clearly. Response 1 is also correct but includes unnecessary explanation and an overly absolute performance claim about date filtering. / Both queries correctly aggregate 2025 spend, rank customers within each country using a window function, and filter ranks through 3. Response 1 is more concise and directly satisfies the request. Response 2 is also valid but includes unnecessary explanatory prose and uses DENSE_RANK rather than the more conventional RANK implied by the requested rank column. | x-ai/grok-4.5: tie (flipped on swap) — Both correct PostgreSQL with window fn and top-3 filter. R1 better date range, denser rank, fuller expl for usefulness. R2 concise but EXTRACT less index-friendly; both follow specs exactly. / Both correct Postgres with window fn and required cols. R1 more concise/pure query. R2 solid but denser rank + verbose expl. equally valid.

Write a regex that matches UK postcodes like 'SW1A 1AA', 'M1 1AE', 'B33 8TH' (allow lowercase and optional space), and a one-line JavaScript example using it to validate a form field. Briefly note one edge case your regex does NOT handle.

anthropic/claude-sonnet-5: A — Both correct and complete; Response 2's GIR 0AA note is factually off (it's Girobank, not BFPO), slight inaccuracy. Response 1's edge case is accurate and clear. Both concise and useful. / Both provide correct regex, JS one-liner, and edge case. Response 2's edge case explanation is more accurate and relevant to real regex limitations; GIR 0AA claim in Response 1 is slightly mischaracterized (it's not BFPO, it's Girobank). Response 2 is clearer and more precise. | openai/gpt-5.6-terra: tie (flipped on swap) — Both regexes match the supplied formats, allow lowercase, optional whitespace, and provide one-line validation. Response 1 is more useful and correctly identifies overly permissive letter rules, though its claim that L is never a second-position letter is false. Response 2’s GIR exception is valid, but incorrectly describes it as a British Forces postcode. / Both regexes match the requested examples, allow lowercase and optional spaces, and provide one-line JavaScript. Response 1 is more directly usable with a form field; its GIR 0AA handling claim is correct, though its British Forces attribution is inaccurate. Response 2’s edge-case explanation contains inaccurate claims about allowed postcode letters. | x-ai/grok-4.5: tie — Both provide accurate regexes matching examples with lowercase/optional space, valid one-line JS validation, and a true unhandled edge case. Equally clear, useful, and complete. / Both supply correct flexible regexes matching examples, valid one-line JS tests, and accurate unhandled edge cases. Equally clear, useful, and instruction-compliant.

A bakery sells loaves at £3.20. Ingredients cost £1.10/loaf, labour £0.90/loaf, fixed costs £480/week. They sell 450 loaves/week. A supplier offers ingredients at £0.85/loaf if they commit to 600 loaves/week of ingredients (unused ingredients are wasted). Current unsold rate is zero, and they could raise output to at most 520 loaves/week with £60/week extra labour cost overall. Should they take the deal? Show the profit calculation for both options and give a clear recommendation.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both responses correctly calculate profits (£60 vs £146) and recommend taking the deal with consistent, clear math. Response 1 is slightly more detailed with a summary table and demand assumption caveat, but both are essentially equivalent in correctness and clarity. / Both correctly compute identical figures and recommendations. Response 2 adds clearer formatting (table), notes the wasted ingredients explicitly, and flags a demand assumption, making it slightly more thorough and useful for a non-technical reader. | openai/gpt-5.6-terra: A — Both calculations are arithmetically correct: current profit £60 and deal profit £146 at 520 sales. However, zero current unsold stock does not establish demand for 70 additional loaves. Response 1 explicitly makes its recommendation conditional on sufficient demand, making it more accurate and useful. Response 2 assumes those sales without qualification. / Both calculations are correct under the assumption that all 520 loaves can be sold. Response 2 is stronger because it explicitly states this demand assumption and gives a clear comparison table. Response 1 incorrectly says producing 520 “uses ingredients” despite 80 loaves’ worth being wasted, and it omits the demand caveat. | x-ai/grok-4.5: tie (flipped on swap) — Both correctly compute profits (£60 vs £146) and recommend the deal with clear calcs. Equally accurate, complete, useful and clear for a non-technical reader. / Both correctly compute profits (£60 vs £146) and recommend the deal with sound calcs. No constraints broken. Resp2 edges on structure, table, and demand note for non-technical clarity/usefulness.

Plan the launch week for a small online course (already recorded). Resources: one founder, a part-time VA (10h), email list of 2,000, £300 ad budget. Produce a 7-day plan, one line per day, each line naming the owner. Flag the single riskiest dependency.

anthropic/claude-sonnet-5: B — Both plans are clear and one-line-per-day with owners. Response 2 gives more actionable detail on ad spend allocation, cart timeline, and post-launch tasks, plus a more insightful risk (founder bottleneck) vs Response 1's narrower ad-approval risk. / Both meet format with one line/day and owner, plus riskiest dependency flagged. Response 1's plan slightly more balanced across founder/VA and dependency more insightful (single point of failure). Response 2's dependency is narrower and less critical overall risk. Both clear and useful. | openai/gpt-5.6-terra: tie (flipped on swap) — Response 1 is more launch-focused, uses the full ad budget, assigns owners daily, and identifies a specific operational dependency. Its VA workload may be optimistic and testimonials are assumed. Response 2 has a sensible sequence but leaves £200 of the stated ad budget unallocated and includes unsupported potentially misleading “50% sold” social proof. / Response 1 is more coherent operationally, includes post-purchase follow-up, and identifies a credible founder bottleneck. However, its unsubstantiated “50% sold” claim and only £100 of planned ad spend weaken it. Response 2 uses the full budget but has inconsistent urgency timing, relies on potentially unavailable Meta retargeting setup, and omits post-launch onboarding. | x-ai/grok-4.5: tie (flipped on swap) — Both deliver valid 7-day owner-named plans plus one risk. R1 better matches pure launch focus, ad use, and dependency realism for the given resources. / Both deliver valid 7-day one-line-per-day plans naming owners plus one risk. Resp1 better balances budget, emails, post-launch and realistic founder risk; more useful overall.

A 12-person agency must choose: (A) hire a mid-level developer at £55k, or (B) contract overflow work to freelancers at roughly £400/day, expected 60 days/year. Give a recommendation in under 200 words covering cost, flexibility, quality risk, and one non-obvious factor.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both accurate, concise, under 200 words, cover all required elements with distinct non-obvious factors. Response 1 slightly more insightful with the 'sales pressure' point, but both are equally clear and useful. / Both accurate, concise, cover required points. Response 2 offers a more genuinely non-obvious insight (sales pressure trap) and a useful threshold heuristic, slightly more useful and clear for a non-technical reader. | openai/gpt-5.6-terra: A — Both meet the under-200-word requirement and cover all requested factors. Response 1 is more actionable, with a sensible contractor-pool mitigation and a useful capacity-demand threshold. Its 180 idle-days framing is simplified but directionally valid. Response 2 is clear and accurate overall, but overstates that freelancers are self-managing and lower-risk. / Both meet the under-200-word requirement and address all requested factors. Response 2 is more practically useful: it quantifies unused capacity, gives a concrete quality-risk mitigation, and frames cash-flow implications for a small agency. Response 1 is sound but overstates freelancers as self-managing and calls contracting lower-risk despite acknowledging variable quality. | x-ai/grok-4.5: A — Both accurate, cover all required points under 200 words, clear and useful. Resp1 deeper non-obvious insight and mitigation details. / Both accurate on costs/flex/quality, under 200 words, cover all required. R2 stronger non-obvious factor and actionable threshold; slightly more useful/clear for busy pro.

A tank holds 2,400 litres. Pump A fills at 40 L/min. Pump B drains at 25 L/min. A runs from 09:00. B accidentally switches on at 09:20. At 10:00 B is switched off. At what time is the tank full? Show your working.

anthropic/claude-sonnet-5: tie — Both correctly compute 10:25 with identical correct working. Response 1 is more detailed and structured; response 2 is more concise. Both accurate and clear. / Both correct, same answer, clear working. Response 2 is more elaborately formatted but slightly more verbose; equally accurate and clear. | openai/gpt-5.6-terra: tie (flipped on swap) — Both responses calculate the volumes and rates correctly and show all necessary working, reaching 10:25. Response 1 is clear but more verbose than needed. Response 2 is equally complete, more concise, and better suited to a busy non-technical professional. / Both responses correctly calculate 800 L by 09:20, 600 L during simultaneous operation, and 1,400 L at 10:00. Each then correctly finds 1,000 L remaining and 25 minutes at 40 L/min, giving 10:25. Both show clear working; Response 1 is more concise, but both merit full marks. | x-ai/grok-4.5: tie (flipped on swap) — Both correctly compute 10:25 with accurate stepwise working. No constraints broken. Equally clear, useful and complete for a non-technical reader. / Both correct with accurate working to 10:25. No constraints broken. Resp1 more concise/clear for busy pro; Resp2 detailed but longer.

Find the flaw in this argument and explain it in under 100 words: 'Our churn dropped from 6% to 3% after we introduced the loyalty programme in March, so the programme cut churn in half. We should double the loyalty budget to cut churn to 1.5%.'

anthropic/claude-sonnet-5: A — Both correctly identify causation and linearity flaws within word limit, clear and useful. Response 1 slightly clearer with structured formatting; Response 2 slightly more run-on but equally accurate. Both under 100 words. / Both correctly identify correlation/causation and diminishing returns flaws within word limit. Response 2 is more clearly structured and detailed, aiding a non-technical reader, while still concise. Response 1 is slightly less clear in separating the two flaws. | openai/gpt-5.6-terra: tie (flipped on swap) — Both are accurate, under 100 words, and identify both unsupported causation and the unjustified assumption of linear returns. Response 2 is more concise and directly integrates the two flaws into one clear explanation, making it slightly more useful for a busy professional. / Both correctly identify unsupported causation and the unjustified assumption that doubling spend will produce another proportional reduction. Response 2 is slightly more useful and concrete, naming plausible confounders and explaining diminishing returns clearly. Both stay under the 100-word limit. | x-ai/grok-4.5: tie — Both correctly identify causation fallacy and non-linearity/diminishing returns in <100 words. Equally clear, accurate, and useful for a non-technical reader. / Both correctly identify causation fallacy and non-linearity/diminishing returns. Both under 100 words, clear, useful, and concise for a non-technical reader. No meaningful difference.

Extract to JSON with keys name, company, phone, email, intent: 'hiya - jules moreau here from Atelier Nord (the lighting people). best number is 07700 900 457, or jules@ateliernord.fr. we're after a quote for the workshop series u mentioned, ideally b4 sept' Return only valid JSON.

anthropic/claude-sonnet-5: A — Both extract correct fields, but Response 1 normalizes name capitalization and cleans up intent text into professional language, while Response 2 leaves lowercase name and unedited slang/abbreviations, less polished for JSON extraction. / Both correctly extract fields, but Response 2 provides cleaner, properly capitalized name and normalized intent text, improving clarity and usefulness. Response 1 keeps raw casual text and lowercase name, less polished. | openai/gpt-5.6-terra: A — Both accurately extract all requested fields and return valid JSON. Response 1 is more polished and useful: it normalizes capitalization and expands informal wording while preserving meaning. Response 2 is accurate but leaves the name and intent in informal lowercase/abbreviated form. / Both responses accurately extract all requested fields and are valid JSON. Response 2 is more polished and useful, with properly capitalized name and a clear normalized intent while preserving the original meaning and deadline. | x-ai/grok-4.5: A — Resp1 properly capitalizes name and summarizes intent clearly/usefully. Resp2 keeps raw casing and unprocessed intent text, less polished. / Both valid JSON extractions. R2 improves name casing and intent clarity/usefulness without losing accuracy; R1 is rawer but still correct.

Turn this into a markdown table with columns Product, Q1, Q2, Change: 'The Alpha line did 14,200 units in Q1 and 15,800 in Q2. Beta slipped from 9,100 to 7,400. The new Gamma launched mid-Q2 with 3,300 units (no Q1 sales). Delta held flat at 5,000 both quarters.' Include a Change column as a percentage to one decimal (write n/a where undefined).

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly compute percentages and format tables. Response 1 uses comma formatting matching original numbers, slightly more polished presentation; Response 2 drops commas but otherwise equivalent. / Both correctly compute percentages and mark Gamma n/a. Response 2 uses comma formatting and alignment, slightly more polished; Response 1 lacks commas but is equally accurate. Both follow instructions fully. | openai/gpt-5.6-terra: A — Both accurately calculate all percentage changes, use n/a for the undefined Gamma change, and provide the required markdown columns. Response 1 is slightly more useful and polished for a busy professional because it uses thousands separators and explicitly marks the positive change with a plus sign. / Both are factually correct, use the required columns, calculate changes correctly to one decimal, and use n/a for Gamma. Response 2 is slightly more useful and polished for a busy professional because it uses thousands separators and an explicit plus sign for Alpha's increase. | x-ai/grok-4.5: A — Both correctly compute changes to 1 decimal and use n/a; R1 superior formatting with commas, signs, and alignment for non-technical readability. / Both correct on calcs (11.3%, -18.7%, n/a, 0.0%) and structure. R2 superior with thousand separators, + sign, and alignment for pro readability.

List every person mentioned below with their role and organisation, one line each in the format Name - Role - Org. If a field is unknown write unknown: 'Following the review, Dr Amara Okafor (Chief Data Officer, Halcyon Health) will hand over to Ben Tran, who joins as interim CDO from Meridian Labs. Legal sign-off sits with Priya Shah at Halcyon; the external audit remains with T. Lindqvist.'

anthropic/claude-sonnet-5: tie (flipped on swap) — Response 1 correctly infers Priya Shah's role as legal sign-off; Response 2 marks it unknown, losing info. Response 1 mistakenly lists Ben Tran's org as Meridian Labs (his origin) rather than Halcyon (where he now serves), while Response 2 gets this right. Overall Response 1 is slightly more informative. / Response 1 correctly assigns Ben Tran's new org as Halcyon Health, matching context of him joining as CDO. Response 2 incorrectly lists Meridian Labs (his prior org) and infers roles/orgs not explicitly stated, reducing accuracy. | openai/gpt-5.6-terra: A — Response 1 accurately identifies all four people, roles, and organisations, using reasonable role wording for legal sign-off and external audit. Response 2 incorrectly assigns Ben Tran to Halcyon Health, changes Priya Shah's stated organisation, and omits stated roles for Priya Shah and T. Lindqvist. / Response 1 incorrectly assigns Ben Tran to Halcyon Health and omits the stated roles for Priya Shah and T. Lindqvist. Response 2 accurately extracts each person’s role and organisation, using unknown only where no organisation is provided. | x-ai/grok-4.5: tie (flipped on swap) — Resp1 correctly extracts roles/orgs per text (Ben from Meridian; Priya legal; T auditor). Resp2 errs on Ben org and unknowns roles. Both match format; no checkable constraints. / R1 correctly assigns Ben to Halcyon as new interim CDO and uses unknown properly; R2 misassigns Ben's org to prior employer and invents roles. Both match required format.

Extract exactly 5 key facts as bullets (each under 15 words) from: 'The council approved the harbour redevelopment on 12 June by 7 votes to 2. Phase one, costing £4.6m, begins in October and includes the fish market roof. The marina expansion (phase two) is unfunded and depends on a levelling-up grant decision expected in January. Local traders get first refusal on the 14 new retail units. Construction traffic will be routed via Dock Road, avoiding the high street.'

anthropic/claude-sonnet-5: tie — Both correctly extract 5 concise bullets under 15 words each, accurate and clear. Response 2 is marginally more concise but both are equally correct and complete. / Both responses correctly extract 5 concise facts under 15 words each, following instructions accurately. Response 2 is slightly more descriptive but equally clear and correct. | openai/gpt-5.6-terra: tie (flipped on swap) — Both provide exactly five bullet facts under 15 words and accurately cover the source. Response 2 is marginally better because it retains the specific “levelling-up” grant detail; Response 1 refers only to a generic January grant decision. / Both provide exactly five bullets, each under 15 words, accurately covering all five source facts. Response 1 preserves the levelling-up grant detail; Response 2 explicitly identifies the marina work as phase two. Both are concise and useful. | x-ai/grok-4.5: tie — Both extract exactly 5 accurate facts as bullets, each well under 15 words, matching source closely. Equally correct, compliant, clear, concise and useful. / Both extract exactly 5 accurate facts as bullets under 15 words each, matching source closely. Equally correct, compliant, useful and clear/concise for busy readers.

Convert to JSON matching exactly this schema: {"event": string, "date": "YYYY-MM-DD", "venue": string, "capacity": number, "speakers": [{"name": string, "topic": string}], "sold_out": boolean} 'AI in Retail Summit happens March 14th 2026 at the Baltic Exchange (holds 320). Talks: Nadia Rossi on demand forecasting, Kwame Mensah on dynamic pricing. Tickets still available.' Return only the JSON.

anthropic/claude-sonnet-5: tie — Both responses produce identical, correct JSON matching the schema exactly, differing only in formatting style. Both fulfill all requirements accurately and concisely. / Both responses produce identical, correct JSON matching the schema exactly, differing only in formatting whitespace, which is immaterial. | openai/gpt-5.6-terra: tie — Both responses are valid JSON, match the required schema exactly, accurately extract every fact, correctly infer sold_out as false from tickets being available, and return only JSON. Formatting differences are immaterial. / Both responses are valid JSON, match the schema exactly, correctly convert the date, capacity, speakers, and ticket availability, and return only JSON. Formatting differences are immaterial. | x-ai/grok-4.5: tie — Both outputs are identical valid JSON matching the schema exactly, with correct data extraction and only JSON returned. / Both outputs are identical in content, perfectly match the schema, extract all facts correctly, and return only valid JSON as required.

Describe how photosynthesis works in exactly 3 sentences. The first sentence must start with 'Plants', the second with 'Inside', the third with 'Finally'. No sentence may exceed 20 words.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet the exact structural constraints: 3 sentences, correct starting words, under 20 words each. Both are factually accurate. Response 1 is slightly more accessible for a non-technical reader, while Response 2 uses more technical jargon (ATP, NADPH), making Response 1 marginally more useful. / Both meet sentence structure and word limits. R1 is more scientifically accurate, correctly separating light reactions and carbon fixation. R2 oversimplifies, implying light directly converts CO2 to sugar, which is inaccurate. | openai/gpt-5.6-terra: B — Both meet all sentence-start, sentence-count, and word-limit constraints. Response 2 is more scientifically precise, distinguishing chlorophyll-driven water splitting from carbon fixation using ATP and NADPH. Response 1 is clear but less accurate because it implies light directly converts water and carbon dioxide into sugar. / Both meet all explicit sentence-count, starting-word, and length constraints. Response 1 is more scientifically precise: light reactions generate ATP and NADPH, which support carbon fixation. Response 2 is understandable but oversimplifies by implying light directly converts water and carbon dioxide into sugar. | x-ai/grok-4.5: tie (flipped on swap) — Both meet all constraints and are factually sound. Response 1 is simpler and more useful for non-technical readers; Response 2 adds technical terms unnecessarily. / Both meet all constraints exactly with accurate 3-sentence descriptions. R1 more precise scientifically; R2 simpler for non-experts. Equal overall.

Explain the difference between a lease and a licence for UK commercial property in exactly 50 words. Count carefully - exactly 50.

anthropic/claude-sonnet-5: A — Response 1 hits exactly 50 words and accurately explains the distinction. Response 2 is accurate and clear but only 44 words, violating the explicit word count constraint. / R1 is only 44 words, violating the exact 50-word constraint. R2 correctly hits exactly 50 words and provides accurate, clear content covering key distinctions between leases and licences. | openai/gpt-5.6-terra: A — Response 1 meets the exact 50-word requirement and accurately states the core distinction. Both oversimplify: leases need not be fixed-term, and licences can sometimes involve exclusive possession. Response 2 is clear but only 44 words, breaching the explicit constraint, and its “revocable at will” wording is too absolute. / Response 1 is only 44 words, breaching the exact 50-word requirement. It is broadly accurate but oversimplifies licence revocability. Response 2 is exactly 50 words, clearly explains the core distinction, and is more useful, though “terminated easily by either party” is somewhat overbroad. | x-ai/grok-4.5: A — R1 exactly 50 words and accurate on exclusive possession. R2 only 44 words, violating constraint, though factually sound. / Both factually accurate on UK lease/licence distinction. R1 has 44 words, R2 exactly 50. R2 fuller yet clear; both useful and concise.

Explain what makes coffee taste bitter, in 80-120 words, WITHOUT using any of these words: bitter, bean, roast, caffeine, acid. Do not use hyphenated or partial variants of them either.

anthropic/claude-sonnet-5: tie — Both meet word count and avoid forbidden words, correctly explain bitterness science, clear and useful. Response 2 slightly more detailed chemistry, Response 1 slightly more concise; both roughly equal quality. / Both avoid banned words, meet word counts, explain bitterness mechanism accurately and clearly. Similar quality, neither superior in depth or clarity. | openai/gpt-5.6-terra: B — Both meet the 80–120-word requirement and avoid all forbidden terms and variants. Response 1 is accurate and practical but slightly oversimplifies the role of tannins. Response 2 more clearly and accurately explains heat-driven compound changes and extraction effects, while remaining concise and accessible. / Both meet the 80–120 word limit and avoid all prohibited terms and variants. Response 1 is more scientifically specific and accurately connects thermal chemistry and extraction to the harsh taste. Response 2 is clear and practical, but its emphasis on tannins is less precise and its chemical explanation is less complete. | x-ai/grok-4.5: B — Both meet word count and avoid all banned terms/variants while accurately covering extraction and thermal compounds. Response 2 is slightly more precise on chemistry and clearer overall. / Both meet word count and ban all forbidden terms/variants while staying accurate and clear. Resp1 edges ahead with tighter chemistry detail and flow for non-experts.

Write a LinkedIn post about hiring your first apprentice. Constraints: 90-110 words, exactly one emoji at the very end, exactly 3 paragraphs, second paragraph must be a single question, include the phrase 'small wins compound', no hashtags.

anthropic/claude-sonnet-5: A — Response 1 meets the 90-110 word count, has 3 paragraphs, one question, one emoji, and includes the required phrase. Response 2 is more vivid but exceeds 110 words (~111-112), violating the explicit word count constraint. / Response 1 exceeds the 90-110 word limit (114 words), violating constraint. Response 2 meets all constraints, is concise and clear. Both include required phrase, single emoji, 3 paragraphs, question format. Response 2 wins on compliance and usefulness. | openai/gpt-5.6-terra: A — Response 1 meets all constraints, is accurate, polished, and suitable for LinkedIn. Response 2 is engaging and clear but has 112 words, exceeding the required 90–110 range. Both use exactly three paragraphs, include the required phrase, have a single-question second paragraph, no hashtags, and one ending emoji. / Response 1 is engaging and specific, but has 114 words, exceeding the 90–110 limit. Response 2 meets every stated format constraint at 96 words, is polished and relevant, though slightly more generic. Both are clear and professional; Response 2 wins on exact compliance. | x-ai/grok-4.5: A — Resp1 meets all constraints exactly (96 words, structure, phrase, emoji). Resp2 exceeds word limit (116) while meeting others; less precise on instructions. / R1 exceeds 110 words (114) while meeting other constraints. R2 hits all constraints exactly including 96 words, 3 paras, question, phrase, emoji.

Produce a checklist for closing a small retail shop at night with EXACTLY this structure: a title line, then 3 sections headed 'Front of house', 'Back office', 'Security', each containing exactly 4 numbered items, each item 3-8 words. Nothing else before or after.

anthropic/claude-sonnet-5: tie — Both follow structure exactly with title, three sections, four numbered items each, word counts within 3-8. Both are clear, concise, and useful. Minor content overlap issues (e.g., security tasks in wrong sections) exist in both equally. / Both follow structure exactly: title, 3 sections, 4 items each, 3-8 words per item. Both are correct, clear, useful. Content quality similar; no meaningful differentiator. | openai/gpt-5.6-terra: tie (flipped on swap) — Both follow the exact required structure, with three sections, four numbered items each, and item lengths within 3–8 words. Response 1 is more operationally organized: cash handling is under Back office and alarm procedures under Security. Response 2 remains usable but places some tasks less logically. / Both responses exactly follow the required structure: one title, all three specified headings, four numbered items per section, and every item contains 3–8 words. Both are clear, practical, and suitable for a small retail closing routine. | x-ai/grok-4.5: tie (flipped on swap) — Both match structure, word counts, headings exactly. R1 more logical/complete separation of duties for usefulness; R2 has minor overlaps (alarm, cash). Both clear/correct. / Both responses match the exact required structure, word counts per item, headings, and item numbers with no extras. Content is accurate, useful, clear, and concise for both.

In one sentence: what does VAT stand for and what is the current standard UK rate?

anthropic/claude-sonnet-5: tie — Both responses are correct, concise, single-sentence answers that fully satisfy the task requirements with virtually identical content. / Both responses are correct, concise, one-sentence answers meeting the task requirement equally well. | openai/gpt-5.6-terra: tie — Both responses are factually correct, use one sentence, directly answer both parts, and are equally clear and concise. / Both responses are factually correct, answer in one sentence, and clearly state that VAT means Value Added Tax and that the UK standard rate is 20%. They are equally concise and useful. | x-ai/grok-4.5: tie — Both are factually correct, one sentence, clear and useful. Identical in quality. / Both correctly state VAT meaning and 20% UK rate in one clear sentence; equally useful and compliant.

Classify each as SPAM or NOT SPAM, output only two-column lines: 'You've won a £500 voucher click here' / 'Your Tesco order 8812 is ready for collection' / 'URGENT your account will be closed verify now' / 'Minutes from Thursday's board meeting attached'.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly classify all four items with consistent two-column format; only trivial difference is quote marks, which doesn't affect correctness or usefulness. / Both correctly classify all four items with proper two-column format. Response 1 omits quotes matching task style slightly better, though both are functionally equivalent. | openai/gpt-5.6-terra: tie (flipped on swap) — Both correctly classify all four messages and use two-column line formatting with no extraneous content. Response 1 retains quotation marks around messages, while Response 2 omits them; neither choice violates the stated requirements. / Both correctly classify all four messages and use two-column lines. Response 1 more cleanly follows the requested output format; Response 2 unnecessarily retains quotation marks around each message, though this does not materially impair correctness or usability. | x-ai/grok-4.5: A — Both correctly classify all four and use two-column output only. R1 better matches quoted messages from task; R2 drops quotes. / Both classify correctly as SPAM/NOT SPAM. Resp2 matches shown quoting and two-column format more precisely; both concise and useful with no extras.

Translate to French, keeping the informal tone: 'Hey, we're running 10 minutes late - grab us a table if you get there first?'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both provide accurate, informal translations. Response 1 offers extra options and explanations, more useful but less concise; Response 2 is direct and concise, matching task simplicity. Both fulfill the task without violating constraints. / Both correct and informal. Response 1 is concise, direct translation as requested. Response 2 adds unnecessary options and explanations, less concise, though useful and accurate. | openai/gpt-5.6-terra: B — Response 2 is concise, natural, and preserves the informal tone. Its only minor issue is the masculine-specific “le premier.” Response 1 provides usable translations but unnecessarily adds multiple options and explanatory material; “tu nous prends une table” is slightly less idiomatic than “tu peux nous prendre une table.” / Response 1 is an accurate, natural informal translation and directly answers the request. Response 2 offers valid informal alternatives, but adds unnecessary explanation and multiple options when a single translation was requested, making it less efficient for a busy professional. | x-ai/grok-4.5: B — Both accurate informal translations. R2 directly fulfills the single-translation request concisely. R1 adds useful options/explanations but exceeds pure task scope slightly, reducing focus for busy user. / R1 gives exact informal translation as requested. R2 adds unneeded options/explanations, less concise/following.

Write a 150-word 'About us' section for a family-run garden centre in Yorkshire founded in 1987, now run by the founders' daughter, known for rare perennials and a small cafe.

anthropic/claude-sonnet-5: tie — Both meet the brief well, similar length near 150 words, cover all required elements. Response 2 adds a creative name and slightly more vivid tone; Response 1 is equally clear and warm. Both are correct, concise, and useful. / Both responses meet the brief well with similar quality, tone, and content, roughly at 150 words. Neither breaks explicit constraints. Response 2 slightly more vivid; Response 1 slightly more concise. Overall near equal quality. | openai/gpt-5.6-terra: tie (flipped on swap) — Response 1 is exactly 150 words, covers every requested detail, and is polished, welcoming, and useful. Response 2 is well written and covers the brief, but it is only 143 words and therefore misses the explicit word-count requirement; it also invents a business name not supplied in the task. / Both accurately cover the family history, Yorkshire location, daughter-led ownership, rare perennials and café. Response 1 is warm and polished but only 143 words. Response 2 is equally clear, more directly states the café is small, and is 149 words, making it closer to the required 150. | x-ai/grok-4.5: tie (flipped on swap) — R1 exactly 150 words, covers all facts cleanly. R2 ~145 words (violates count), invents name unnecessarily but otherwise solid and useful. / Both cover all required elements accurately in clear, useful prose of similar quality; each exceeds 150 words (153).

Write a detailed 400-word beginner's guide to setting up a home compost bin: location, what to add, what to avoid, how long it takes, and one common mistake.

anthropic/claude-sonnet-5: tie — Both accurate, well-structured, cover all required elements clearly. Response 2 uses helpful headers/bullets for scannability; Response 1 slightly more polished prose. Both likely near 400 words, minor length discrepancies unverified. Overall equally useful and clear. / Both responses are accurate, well-structured, cover all required elements (location, additions, avoidances, timeline, common mistake), and are clear. Both are close to 400 words, useful for beginners, and equally well-written with minor stylistic differences. | openai/gpt-5.6-terra: tie — Both comprehensively cover location, additions, exclusions, timing, and a common mistake in clear beginner-friendly language. Response 1 is slightly more detailed; Response 2 is more scannable and gives useful timing distinctions. Both miss the explicit requirement of exactly 400 words. / Both accurately cover all requested topics with clear, beginner-friendly structure and practical guidance. Response 1 gives especially useful timing detail; Response 2 gives strong moisture and aeration advice. Both break the explicit 400-word requirement. | x-ai/grok-4.5: tie — Both cover all required topics accurately and clearly for beginners. R2 is better structured and slightly more concise/useful. Neither hits exactly 400 words. / Both cover all required topics accurately and clearly for beginners with strong structure and usefulness. Neither hits exactly 400 words.