Gemini 3.5 Flash wins 4–3

Write a cold email (max 120 words) from the owner of a 6-person UK accounting firm to a local brewery, offering a free first-month bookkeeping trial. Friendly but professional, one clear call to action, no buzzwords.

anthropic/claude-sonnet-5: tie — Both meet word limit, tone, and CTA requirements. Response 2 has slightly stronger personalization and clearer CTA (specific call ask), Response 1 slightly more concise. Both well within 120 words, no buzzwords. / Both meet word limit, tone, and CTA requirements clearly. Response 1 adds a nice personal touch; Response 2 is slightly more informative about services. Both concise, professional, no buzzwords. Essentially equal quality. | openai/gpt-5.6-terra: tie (flipped on swap) — Both meet the length limit, offer a free first month, use a friendly professional tone, avoid obvious buzzwords, and include one clear CTA. Response 1 states ownership more explicitly and is slightly tighter. Response 2 feels more personalised and local, but “accounting team” is marginally less clearly an accounting firm. / Both are under 120 words, friendly, professional, and include one clear call to action. Neither explicitly states the firm is UK-based. Response 2 more clearly establishes that the sender is the owner and gives a simpler reply-based CTA. Response 1 is strong but is slightly less direct about ownership. | x-ai/grok-4.5: tie (flipped on swap) — Both under 120w, friendly-pro, one CTA, no buzzwords. R1 tighter, clearer VAT/UK fit, stronger usefulness. / Both meet all criteria: under 120 words, friendly-professional tone, clear free-trial offer, single CTA, no buzzwords. Equally correct, useful, and clear.

Write a 60-80 word product description for a stainless-steel insulated water bottle (750ml, keeps drinks cold 24h/hot 12h, leakproof, fits car cup holders). Target audience: gym-goers. Avoid cliches like 'stay hydrated in style'.

anthropic/claude-sonnet-5: tie — Both meet word count (71 and 69), avoid the cliché, cover all required features, and are clear and appealing. Quality and style are comparable, with no significant differentiator. / Both responses meet word count (60-80), avoid the banned cliché, cover all required features, and are clear, engaging descriptions targeted at gym-goers. Quality and usefulness are essentially equivalent. | openai/gpt-5.6-terra: A — Both meet the 60–80-word requirement and avoid the cited cliché. Response 1 accurately covers every feature, clearly targets gym-goers, and reads naturally. Response 2 is polished but adds an unsupported claim about fitting treadmill docks and uses less natural phrasing (“it tosses safely” and “clean gear”). / Both meet the 60–80-word requirement and avoid the cited cliché. Response 2 clearly covers every product feature in gym-focused, practical language. Response 1 is strong but adds an unsupported claim about fitting treadmill docks and uses slightly more promotional phrasing. Response 2 is more directly useful and precise. | x-ai/grok-4.5: tie (flipped on swap) — Both hit 60-80 words, cover all specs, target gym-goers, avoid banned cliche. R1 slightly tighter and more precise on features without extras. / Both hit 60-80 words, cover all specs, target gym-goers effectively without cliches. Equally accurate, clear, concise and useful.

Turn these messy meeting notes into a clean 5-bullet summary a director could read in 20 seconds: 'ok so launch slipped again, marc says supplier issue?? maybe 3 wks. sarah wants to push the email campaign anyway but tom thinks wait. budget - we're 8k over, mostly the packaging redo. Q: do we tell retail partners now or after new date confirmed. also NEED to hire the warehouse temp before august rush. next mtg tues.'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both accurately capture all key points in 5 clear bullets. Response 1 includes next meeting date, response 2 omits it but has slightly better labeling. Both equally clear and concise; minor tradeoffs balance out. / Both are accurate, concise 5-bullet summaries. Response 2 includes the next meeting date, capturing all details, while Response 1 omits it, making Response 2 slightly more complete and useful. | openai/gpt-5.6-terra: A — Both provide exactly five clear, accurate bullets suitable for a director. Response 1 is stronger because it preserves all key details, including stakeholder positions and the next meeting date. Response 2 is concise and accurate but omits the Tuesday meeting detail and adds an unnecessary introductory sentence. / Both provide exactly five clear, accurate bullets suitable for a director. Response 2 is stronger because it preserves all key details, including attribution for the launch estimate and the next-meeting date. Response 1 omits the Tuesday meeting detail and adds unnecessary introductory text. | x-ai/grok-4.5: A — R1 fully covers all notes incl. next mtg in clean 5 bullets. R2 omits next mtg, adds unnecessary intro. Both useful/clear but R1 more complete/accurate. / Both deliver accurate 5-bullet summaries covering key points cleanly and usefully. R2 is slightly more complete by including the next meeting and naming sources.

Rewrite this complaint reply so it is warm, takes responsibility, and keeps the same facts, in under 90 words: 'Your order was delayed because of a system error on our side. The refund has been processed and will arrive in 3-5 days. We cannot offer further compensation. Let us know if you have questions.'

anthropic/claude-sonnet-5: tie — Both responses accurately preserve facts, take responsibility warmly, and stay under 90 words. Nearly identical quality, clarity, and tone; no meaningful difference in usefulness or correctness. / Both responses are warm, take responsibility, preserve facts, and stay under 90 words. Nearly identical quality in tone, clarity, and usefulness; no meaningful difference to prefer one over the other. | openai/gpt-5.6-terra: A — Both are warm, accountable, clear, and under 90 words. Response 1 preserves the stated 3–5-day timeframe and all core facts. Response 2 changes this to “3–5 business days” and adds “in your account,” which slightly alters the original information. Response 1’s “right now” is a minor softening but less consequential. / Both are warm, accountable, clear, and under 90 words. Response 1 changes “3–5 days” to “3–5 business days,” altering a stated fact. Response 2 retains the timeframe and is concise, though “right now” slightly softens the definitive compensation limitation. | x-ai/grok-4.5: A — Both warm, responsible, under 90 words, keep core facts. R1 more precise on timing/details and concise; R2 slightly alters facts with 'business days' and extras. / Both under 90 words, warm, take responsibility. R1 alters facts slightly (adds 'business' days, 'account'). R2 keeps exact facts, clearer/more concise.

Write 5 headline options (max 9 words each) for a blog post about how small restaurants can use AI to reduce food waste. Mix: 2 practical, 2 curiosity-driven, 1 with a number.

anthropic/claude-sonnet-5: B — R2 correctly follows the 9-word max for all headlines and self-labels word counts accurately. R1's first headline exceeds the 9-word limit (10 words), violating the explicit constraint. Both meet the mix requirement, but R2 is more precise and useful. / Both meet constraints and word counts; Response 1 explicitly labels word counts, confirming compliance, and headlines are slightly punchier. Response 2 is solid but curiosity headlines less compelling. | openai/gpt-5.6-terra: A — Response 1 meets the requested mix, stays within nine words per headline, and is clear and relevant. Response 2 also has a strong mix and useful headlines, but its parenthetical word-count labels are part of each listed option, causing all five to exceed the nine-word maximum literally. / Both provide exactly five headlines, meet the mix requirements, and keep every headline within nine words. Response 2 is slightly stronger because every option explicitly targets small restaurants and food waste more consistently. Response 1’s “small kitchens” phrasing is somewhat less directly aligned with the requested audience. | x-ai/grok-4.5: tie — Both deliver exactly 5 valid headlines (all ≤9 words) with the required 2+2+1 mix, on-topic and clear. Minor style differences only; equally useful and compliant. / Both deliver exactly 5 headlines under 9 words with the required mix and labels. Equally accurate, clear, and useful.

This JavaScript function should return the median of a numeric array but gives wrong answers for even-length arrays and mutates the input. Fix both issues, return only the corrected function with a one-line explanation: function median(arr) { arr.sort(); const mid = Math.floor(arr.length / 2); return arr[mid]; }

anthropic/claude-sonnet-5: tie — Both correctly fix mutation and even-length median with correct comparator, each with a concise one-line explanation. Functionally identical, differing only in formatting. / Both responses provide identical correct logic fixing mutation and even-length median, with concise one-line explanations. Functionally equivalent, differing only in formatting of comment vs bold text. | openai/gpt-5.6-terra: tie — Both correctly avoid mutating the input, sort numerically, and average the two middle values for even-length arrays. Each provides the corrected function and a one-line explanation. Response 2 is formatted slightly differently but is equally clear and compliant. / Both correctly avoid input mutation via copying, sort numerically, and average the two middle values for even lengths. Each provides only the corrected function and a one-line explanation. Response 2 is slightly more compact, but both fully satisfy the task. | x-ai/grok-4.5: A — Both correctly fix mutation via copy, numeric sort, and even-length average. Both supply function + one-line explanation as required. Resp1 slightly clearer/more concise. / Both fix mutation via copy, numeric sort, and even-length average correctly. Resp2 alone returns solely the function plus a true one-line explanation as required.

Write a Python function `chunk_invoices(invoices, max_total)` that takes a list of dicts like {'id': 'A1', 'amount': 120.5} and groups them into batches where each batch's summed amount does not exceed max_total. A single invoice larger than max_total goes in its own batch. Preserve order. Include 3 test cases using assert.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both implementations correctly handle oversized invoices and normal batching, with valid test cases. Response 1's logic is slightly cleaner and explicitly separates oversized invoice handling. Response 2's approach also works correctly but is marginally less explicit. Both pass their asserts. Response 1 is very slightly more robust and clear. / Both implementations correctly handle batching and oversized invoices, with passing self-consistent tests. Response 1's logic is slightly simpler and more readable, while Response 2's large-invoice handling is functionally correct but less clear due to the split logic before appending. | openai/gpt-5.6-terra: B — Both implementations correctly preserve order and isolate oversized invoices. Response 1 is clearer and tests normal, oversized, exact-fit, and empty-input behavior, but it contains four assert statements despite requesting three test cases. Response 2 has exactly three asserts and is correct, but its comments are more verbose and its tests cover fewer edge cases. / Both implementations preserve order, correctly split ordinary positive amounts, and handle oversized invoices. Response 2 explicitly isolates oversized invoices and is more concise. Response 1 relies on later invoices exceeding an oversized running total to isolate it, which is less direct. Response 2 contains four assert cases despite requesting three. | x-ai/grok-4.5: tie — Both correctly implement order-preserving batching with oversized singles alone, plus valid asserts. Clear, complete, useful code; no constraints broken. / Both implement correct order-preserving batching with large-invoice handling and valid asserts. Clear, complete, useful code; no constraints breached.

Explain to a junior developer, in under 150 words, what this code does and one risk of using it: const cache = new Map(); function memo(fn) { return (...args) => { const k = JSON.stringify(args); if (!cache.has(k)) cache.set(k, fn(...args)); return cache.get(k); }; }

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly explain memoization and word count is under 150. Response 2 identifies a more critical, specific bug (shared cache causing wrong results) with a clear example, more useful than the generic memory leak risk in Response 1. / Both correctly explain memoization and give valid, distinct risks (shared cache bug vs memory leak), both under 150 words, clear and useful. Response 1's risk is more specific to this exact code's design flaw; Response 2's is more universally applicable to memoization. Roughly equal quality. | openai/gpt-5.6-terra: tie (flipped on swap) — Both are accurate, clear, and under 150 words. Response 1 correctly identifies unbounded cache growth as a risk. Response 2 identifies the more immediate correctness risk: all wrappers share a cache, so identical arguments across different functions can return another function’s result. Its concrete example is especially useful. / Both accurately explain memoization, JSON-based argument keys, and caching behavior in under 150 words. Response 1 identifies the shared-cache collision between different wrapped functions with a clear example. Response 2 correctly identifies unbounded cache growth and resulting memory risk. Both are clear and useful. | x-ai/grok-4.5: B — Both accurately explain memoization under 150 words with valid risks. Resp2's shared-cache risk is more precise/critical with clear example; slightly higher usefulness/clarity. / Both accurate, under 150 words, clear for juniors. R1 better highlights the shared-cache flaw unique to this impl with example; R2 solid on memory but less precise.

Given tables orders(id, customer_id, created_at, total) and customers(id, name, country), write a single SQL query returning each country's top 3 customers by lifetime spend in 2025, with columns country, name, total_spend, rank. Use a window function. Standard PostgreSQL.

anthropic/claude-sonnet-5: B — Both queries are correct and functional. Response 2 uses date-range filtering (better index usage) and provides helpful explanation for non-technical professionals, but DENSE_RANK could return more than 3 rows on ties, deviating slightly from typical 'top 3' expectation. Response 1's ROW_NUMBER strictly gives 3. Both are close; Response 2's clarity edges out. / Both correctly answer the task with valid PostgreSQL window function queries. Response 1 uses sargable date range filtering (better practice) and provides clear explanation, aiding a non-technical professional. Response 2 uses EXTRACT which is less index-friendly but still correct. Response 1 is more thorough and instructive. | openai/gpt-5.6-terra: A — Response 1 correctly aggregates 2025 spend, uses a window function, and returns exactly three customers per country via ROW_NUMBER. Response 2 is valid PostgreSQL and uses a more index-friendly date filter, but DENSE_RANK can return more than three customers when ties occur, making it less exact for “top 3 customers.” / Both use valid PostgreSQL window-function queries and return the requested columns. Response 2 more precisely returns at most three customers per country via ROW_NUMBER and is concise. Response 1's DENSE_RANK may return more than three customers when ties occur, which can conflict with “top 3 customers,” and includes unnecessary explanation. | x-ai/grok-4.5: tie (flipped on swap) — Both correct Postgres queries with window fn for top-3 per country. R2 slightly better date filter + denserank + useful notes; R1 cleaner/minimal. / Both correct with window fn and proper 2025 filter/top-3. R2 cleaner/more concise for busy user; R1 verbose but explanatory. No constraints broken.

Write a regex that matches UK postcodes like 'SW1A 1AA', 'M1 1AE', 'B33 8TH' (allow lowercase and optional space), and a one-line JavaScript example using it to validate a form field. Briefly note one edge case your regex does NOT handle.

anthropic/claude-sonnet-5: tie — Both provide correct, functionally equivalent regex, JS one-liners, and valid edge case notes. Response 2 is slightly more detailed on the edge case, but both meet requirements well. / Both provide correct, functionally equivalent regexes, valid JS one-liners, and plausible edge cases. Response 2 doesn't trim input but that's minor. Both fulfill instructions well; quality is comparable. | openai/gpt-5.6-terra: tie (flipped on swap) — Both provide a case-insensitive regex, optional space, one-line JavaScript validation, and an edge case. Response 1 accurately identifies omitted position restrictions and GIR 0AA. Response 2 is usable, but its claim that L and O are never second-position letters is inaccurate (for example, OL is a valid area), weakening factual accuracy. / Both regexes match the examples, lowercase, and an optional separator. Response 2 is more accurate and usefully identifies positional restrictions and GIR 0AA, though it provides two edge cases despite requesting one. Response 1 incorrectly claims L is never a second postcode-area letter; valid areas include AL and BL. | x-ai/grok-4.5: tie — Both provide accurate basic UK postcode regexes covering examples with case-insensitivity and optional space, valid one-line JS validators, and relevant unhandled edge cases. Equally clear, useful, and complete. / Both provide accurate regex matching examples with /i and optional space, valid one-line JS, and real edge cases. Equally clear, concise, and useful.

A bakery sells loaves at £3.20. Ingredients cost £1.10/loaf, labour £0.90/loaf, fixed costs £480/week. They sell 450 loaves/week. A supplier offers ingredients at £0.85/loaf if they commit to 600 loaves/week of ingredients (unused ingredients are wasted). Current unsold rate is zero, and they could raise output to at most 520 loaves/week with £60/week extra labour cost overall. Should they take the deal? Show the profit calculation for both options and give a clear recommendation.

anthropic/claude-sonnet-5: B — Response 2 correctly scales labour cost with the increased output (520×£0.90+£60), giving accurate £146 profit. Response 1 fails to scale labour with higher output, incorrectly using £405+£60, inflating profit to £209. Response 2 also provides clearer scenario comparisons and a helpful summary table. / Response 1 correctly scales labour cost with increased output (520×0.90+£60), giving accurate £146 profit. Response 2 incorrectly uses original 450-loaf labour cost +£60, understating costs and overstating profit (£209), a factual error. Both are clear and well-structured, but Response 1 is more accurate. | openai/gpt-5.6-terra: B — Response 1 incorrectly holds base labour at £405 when output rises to 520; variable labour should be 520×£0.90 plus the £60 extra, giving £146 profit, not £209. Response 2 calculates this correctly and appropriately conditions its recommendation on selling the additional 70 loaves. / Response 1 correctly applies £0.90 labour to all 520 loaves plus the £60 overall extra labour cost, giving £146 profit, and appropriately makes the recommendation conditional on selling 520. Response 2 incorrectly holds base labour at £405, omitting £63 labour for the additional 70 loaves; its £209 profit and unconditional recommendation are therefore wrong. | x-ai/grok-4.5: B — Resp1 miscalculates labour (omits per-loaf on extra units) yielding wrong £209 profit; Resp2 correctly applies 520×0.90+60 for accurate £146 and fuller scenarios. / R1 correctly scales labour to 520*0.90+60=528 yielding £146 profit; R2 errs by using only original labour+60=465 for false £209. Both show calcs and recommend yes-if-max-output. R1 clearer/more complete.

Plan the launch week for a small online course (already recorded). Resources: one founder, a part-time VA (10h), email list of 2,000, £300 ad budget. Produce a 7-day plan, one line per day, each line naming the owner. Flag the single riskiest dependency.

anthropic/claude-sonnet-5: B — Response 2 assigns single owner per day as instructed, while Response 1 lists dual owners each day, violating the 'one line, owner' clarity intent. Both are factually sound; Response 2's dependency flag is more insightful and format cleaner. / Task requires one owner per line; Response 1 assigns single owner per day, Response 2 lists dual owners each day, violating 'one owner' instruction. Both flag plausible risks, Response 1 slightly clearer and more concise. | openai/gpt-5.6-terra: tie (flipped on swap) — Both provide seven owner-labelled daily actions and one clear risk. Response 1 is coherent but likely overuses the VA’s 10 hours and assumes a past student. Response 2 uses the VA more plausibly and identifies a critical checkout risk, but its Day 6 “final-day” emails conflict with closing on Day 7 and assumes retargeting setup. / Response 1 is more operationally focused, assigns clear single owners, uses the limited budget sensibly, and identifies a highly credible launch-critical dependency. Response 2 is workable but repeatedly assigns both people, makes the VA's 10-hour limit less credible, and includes lower-priority tasks such as DMs. | x-ai/grok-4.5: B — Both meet format (7 one-line days naming owners + one risk). R2 better allocates scarce VA hours, more realistic risk, clearer ownership split, higher usefulness. / Both deliver valid 7-day plans with owners and a flagged risk. R1 is clearer, assigns single realistic owners matching the 10h VA limit, and identifies a stronger risk. R2 overloads the VA daily.

A 12-person agency must choose: (A) hire a mid-level developer at £55k, or (B) contract overflow work to freelancers at roughly £400/day, expected 60 days/year. Give a recommendation in under 200 words covering cost, flexibility, quality risk, and one non-obvious factor.

anthropic/claude-sonnet-5: tie — Both correctly analyze cost, flexibility, quality risk, and offer a distinct non-obvious factor within word limit. Response 2's scheduling friction point is slightly more insightful and concise; Response 1's management bandwidth point is also strong. Both well within 200 words, clear formatting. / Both under 200 words, cover all required points with similar quality and cost analysis. Response 2's non-obvious factor (management bandwidth) is slightly more distinct/insightful than Response 1's scheduling friction, but both are valid and well-argued. | openai/gpt-5.6-terra: B — Both are under 200 words, cover all requested areas, and calculate freelancer cost correctly. Response 2 is more balanced and useful: its scheduling/availability risk is genuinely non-obvious and tempers the recommendation. Response 1 is strong but overstates freelancers as “plug-and-play” and its £70k employment-cost estimate is less clearly supported. / Both meet the under-200-word requirement and cover all requested factors. Response 1 is more concise and its scheduling-friction point is particularly relevant to sporadic 60-day demand. Response 2 offers stronger quality controls, but overstates freelancers as plug-and-play and free of management overhead. | x-ai/grok-4.5: tie — Both under 200 words, cover all required topics accurately with sound recommendation for B, clear structure, useful non-obvious factors. Equally strong on all criteria. / Both correctly calculate costs, cover all four required topics concisely under 200 words, and give practical advice suitable for a non-technical reader. Equally clear and useful.

A tank holds 2,400 litres. Pump A fills at 40 L/min. Pump B drains at 25 L/min. A runs from 09:00. B accidentally switches on at 09:20. At 10:00 B is switched off. At what time is the tank full? Show your working.

anthropic/claude-sonnet-5: tie — Both responses are correct, clear, and well-structured with identical calculations and final answer of 10:25. Response 2 uses LaTeX formatting which may be slightly less readable for a non-technical audience but is otherwise equivalent. / Both responses are correct, clear, well-structured, and reach the same accurate answer (10:25) with identical calculations. No meaningful difference in quality. | openai/gpt-5.6-terra: tie — Both responses correctly calculate 800 L by 09:20, then 600 L during the 40-minute net-fill period, leaving 1,000 L at 10:00. At 40 L/min, this takes 25 minutes, giving 10:25. Both show clear working, follow instructions, and are accessible; Response 1 is marginally plainer but not enough to distinguish scores. / Both responses correctly calculate 800 L by 09:20, 600 L added while both pumps run, 1,400 L at 10:00, and 25 further minutes at 40 L/min. Both show clear working, follow all instructions, and reach 10:25. Response 2 is marginally more concise, but not enough to distinguish scores. | x-ai/grok-4.5: tie — Both responses correctly calculate intervals, net rates, and arrive at 10:25 with clear working. Identical accuracy, structure, and usefulness; no constraints violated. / Both responses correctly calculate intervals and arrive at 10:25 with clear working. Identical accuracy, structure, and usefulness; no constraints violated.

Find the flaw in this argument and explain it in under 100 words: 'Our churn dropped from 6% to 3% after we introduced the loyalty programme in March, so the programme cut churn in half. We should double the loyalty budget to cut churn to 1.5%.'

anthropic/claude-sonnet-5: tie — Both responses correctly identify the same two flaws (correlation/causation and linear extrapolation/diminishing returns) with nearly identical clarity, accuracy, and concision. Both are under 100 words and equally useful to a non-technical reader. / Both responses correctly identify the same two flaws (correlation/causation and diminishing returns/linear extrapolation) with similar clarity and concision, staying under 100 words. Nearly identical quality and content. | openai/gpt-5.6-terra: tie — Both accurately identify unsupported causal attribution and unjustified linear budget extrapolation, stay under 100 words, and are clear for non-technical readers. Response 1 adds useful nuance about a churn floor; Response 2 is slightly more concise. Neither explains how to test causality, but that was not required. / Both accurately identify the unsupported causal claim and the unjustified assumption that doubling spend will halve churn again. Both are clear, concise, under 100 words, and useful to a non-technical professional. Response 2 is slightly more specific, but neither has a material advantage. | x-ai/grok-4.5: tie — Both correctly identify correlation/causation and non-linear scaling flaws, stay under 100 words, and are clear/useful for non-experts. Nearly identical quality. / Both correctly identify causation and non-linearity flaws, stay under 100 words, are clear, concise, and useful. Nearly identical quality.

Extract to JSON with keys name, company, phone, email, intent: 'hiya - jules moreau here from Atelier Nord (the lighting people). best number is 07700 900 457, or jules@ateliernord.fr. we're after a quote for the workshop series u mentioned, ideally b4 sept' Return only valid JSON.

anthropic/claude-sonnet-5: B — Both extract data correctly, but Response 1 wraps JSON in markdown code fences, violating the 'return only valid JSON' instruction. Response 2 is clean JSON with slightly better formatting (proper case, expanded 'before September'). / Response 1 returns clean JSON as requested with proper name capitalization. Response 2 wraps JSON in markdown code fences, violating 'return only valid JSON' instruction, and uses lowercase name and informal abbreviation 'b4 sept' in intent. | openai/gpt-5.6-terra: B — Response 1 accurately extracts all fields, but markdown code fences mean it does not return only valid JSON. Response 2 is valid JSON, accurately preserves contact details, standardizes capitalization, and clearly expands the abbreviated timing request without changing its meaning. / Response 1 is valid JSON, accurately extracts all fields, and clearly expands the intent. Response 2 extracts the information accurately but violates the requirement to return only valid JSON because it includes Markdown code fences. | x-ai/grok-4.5: B — R1 wraps JSON in markdown code fence, violating 'only valid JSON'. R2 is pure valid JSON with accurate extraction, proper casing, and clearer intent phrasing. Both factually sound; R2 superior on format and polish. / Resp1 is pure valid JSON with accurate polished fields. Resp2 wraps in markdown (violates only-JSON) and keeps raw casing/abbrev intent.

Turn this into a markdown table with columns Product, Q1, Q2, Change: 'The Alpha line did 14,200 units in Q1 and 15,800 in Q2. Beta slipped from 9,100 to 7,400. The new Gamma launched mid-Q2 with 3,300 units (no Q1 sales). Delta held flat at 5,000 both quarters.' Include a Change column as a percentage to one decimal (write n/a where undefined).

anthropic/claude-sonnet-5: tie — Both correctly compute percentages and format tables identically in content; Response 2 adds a brief intro and plus sign, minor stylistic difference, both equally accurate and clear. / Both accurately compute percentages and format tables correctly; Response 1 adds sign for clarity but is otherwise identical in substance. Both meet instructions equally well. | openai/gpt-5.6-terra: tie — Both responses accurately extract all quarterly values, calculate percentage changes correctly to one decimal, use n/a for the undefined Gamma change, and provide the required markdown table columns. Response 2 adds a harmless introductory sentence and explicit positive sign; neither reduces usefulness or clarity. / Both responses accurately extract all quarterly figures, calculate percentage changes correctly to one decimal, use n/a for undefined Gamma growth, and provide the requested markdown table columns. Response 1’s plus sign for Alpha is acceptable and improves readability. | x-ai/grok-4.5: tie (flipped on swap) — Both correctly compute changes to 1 decimal, use n/a, and produce valid markdown tables matching required columns. Minor style differences only; equally accurate, compliant, clear and useful. / Both tables accurate with correct % calcs and n/a. Resp1 clearer with + sign and alignment; Resp2 omits + and has minimal formatting. No constraints broken.

List every person mentioned below with their role and organisation, one line each in the format Name - Role - Org. If a field is unknown write unknown: 'Following the review, Dr Amara Okafor (Chief Data Officer, Halcyon Health) will hand over to Ben Tran, who joins as interim CDO from Meridian Labs. Legal sign-off sits with Priya Shah at Halcyon; the external audit remains with T. Lindqvist.'

anthropic/claude-sonnet-5: tie — Both responses are nearly identical, correctly listing all four people with roles and orgs, using 'unknown' where appropriate. Minor wording difference (Legal sign-off vs Legal sign-off) negligible; both follow instructions well. / Both responses are nearly identical, correctly listing all four persons with roles and orgs in the specified format. Minor difference in wording ('External Auditor' vs 'External audit') doesn't affect accuracy or usefulness. | openai/gpt-5.6-terra: A — Response 1 accurately preserves the stated role wording for every person and uses unknown correctly. Response 2 is also complete and correctly formatted, but changes the stated role “External audit” to “External Auditor,” adding a more specific title not explicitly provided. / Both identify all four people and provide the requested three-field, one-line format. Response 2 most faithfully preserves “External audit” as T. Lindqvist’s stated role. Response 1’s “External Auditor” is a plausible interpretation but slightly changes the wording and implies a specific job title not explicitly given. | x-ai/grok-4.5: tie (flipped on swap) — Both correctly extract all four people in required format with accurate roles/orgs; minor phrasing diff on Lindqvist is immaterial. / Both extract all four people accurately in required format. Resp1 better phrases T. Lindqvist role as External Auditor vs External audit. Minor org abbreviation identical. No constraints broken.

Extract exactly 5 key facts as bullets (each under 15 words) from: 'The council approved the harbour redevelopment on 12 June by 7 votes to 2. Phase one, costing £4.6m, begins in October and includes the fish market roof. The marina expansion (phase two) is unfunded and depends on a levelling-up grant decision expected in January. Local traders get first refusal on the 14 new retail units. Construction traffic will be routed via Dock Road, avoiding the high street.'

anthropic/claude-sonnet-5: A — Response 1 includes the vote count (7-2), a key fact omitted by Response 2. Both meet word limits and format; Response 1 is slightly more complete and accurate. / Both meet the 5-bullet, under-15-words constraint. Response 2 includes the vote count (7-2) which Response 1 omits, making it slightly more complete and accurate while remaining concise. | openai/gpt-5.6-terra: A — Both provide exactly five bullets, each under 15 words, and accurately summarize the source. Response 1 is stronger because it retains the approval vote count and specifies the levelling-up grant, preserving more key detail. Response 2 omits the 7-2 vote and makes the grant less specific. / Both provide exactly five concise, accurate bullets under 15 words. Response 2 is stronger because it retains the approval vote and specifies the levelling-up grant, preserving more key source details. Response 1 omits those details but remains useful and accurate. | x-ai/grok-4.5: A — Both meet exact 5-bullet <15-word constraints with accurate facts. R1 more complete (includes vote tally, phase two, levelling-up details) hence higher correctness and usefulness. / Both meet exact 5-bullet <15-word constraints and are accurate/clear. Resp2 includes more key details (vote tally, levelling-up) for slightly higher completeness/usefulness.

Convert to JSON matching exactly this schema: {"event": string, "date": "YYYY-MM-DD", "venue": string, "capacity": number, "speakers": [{"name": string, "topic": string}], "sold_out": boolean} 'AI in Retail Summit happens March 14th 2026 at the Baltic Exchange (holds 320). Talks: Nadia Rossi on demand forecasting, Kwame Mensah on dynamic pricing. Tickets still available.' Return only the JSON.

anthropic/claude-sonnet-5: tie — Both responses are identical, correctly formatted JSON matching the schema exactly with accurate data extraction. / Both responses are identical, correct, and fully compliant with the schema and instructions. | openai/gpt-5.6-terra: tie — Both responses exactly match the required JSON schema, accurately convert every supplied detail, use the required date format and boolean value, and contain only valid JSON. No explicit constraint is breached. / Both responses exactly match the required JSON schema and correctly extract all event details. Dates, capacity, speaker topics, and sold-out status are accurate. Both return only valid JSON. | x-ai/grok-4.5: tie — Both responses are identical, perfectly match the schema with accurate data extraction, and return only the JSON as required. / Both responses are identical, perfectly match the schema with accurate data extraction, return only JSON, and fully satisfy all criteria equally.

Describe how photosynthesis works in exactly 3 sentences. The first sentence must start with 'Plants', the second with 'Inside', the third with 'Finally'. No sentence may exceed 20 words.

anthropic/claude-sonnet-5: tie — Both responses correctly follow the 3-sentence structure, starting words, and word limits, and both accurately describe photosynthesis with clear, concise language. Quality and correctness are essentially equivalent. / Both responses meet the 3-sentence structure, correct starting words, and word limits per sentence. Content is accurate and clear in both. Very similar quality overall. | openai/gpt-5.6-terra: tie — Both responses are accurate, clear, and useful summaries of photosynthesis. Each has exactly three sentences, uses the required opening words in order, and keeps every sentence below 20 words. / Both responses use exactly three sentences, required sentence openings, and fewer than 20 words per sentence. Both accurately summarize photosynthesis in clear, accessible language, though each necessarily simplifies the underlying chemical process. | x-ai/grok-4.5: tie — Both meet all constraints exactly with accurate, clear 3-sentence descriptions under 20 words each. Equally useful and concise. / Both responses exactly meet all constraints with accurate, clear, concise explanations suitable for non-experts. No meaningful differences.

Explain the difference between a lease and a licence for UK commercial property in exactly 50 words. Count carefully - exactly 50.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both hit exactly 50 words and are accurate, clear, and concise. Response 2 slightly more precise, noting the proprietary interest binding successors, a key legal distinction, giving it a marginal edge. / Both responses are exactly 50 words, accurate, clear, and cover key distinctions (exclusive possession, proprietary interest vs personal right, statutory protection). Equally useful and concise, with no constraint violations. | openai/gpt-5.6-terra: tie (flipped on swap) — Both are exactly 50 words, accurate at a practical level, and clearly distinguish exclusive possession and proprietary rights from personal permission. Both slightly overgeneralise statutory protection and revocability, which depend on the arrangement and legislation. Response 2 usefully qualifies security of tenure with “often”; Response 1 is marginally simpler. / Both are exactly 50 words, clear, and useful. Response 1 better captures the core distinction: exclusive possession and proprietary versus personal rights. Response 2 is slightly less precise because a lease does not invariably create a legal estate, and statutory renewal/security rights are not universal. | x-ai/grok-4.5: tie — Both exactly 50 words, factually accurate on UK lease vs licence distinctions, clear and useful. No differences warranting preference. / Both hit exactly 50 words, are factually accurate on UK lease vs licence distinctions, clear, concise and useful. No material differences.

Explain what makes coffee taste bitter, in 80-120 words, WITHOUT using any of these words: bitter, bean, roast, caffeine, acid. Do not use hyphenated or partial variants of them either.

anthropic/claude-sonnet-5: tie — Both stay within 80-120 words, avoid banned words/variants, and clearly explain bitterness causes. Response 2 is slightly more specific and scientifically detailed, but both are accurate, clear, and useful. / Both responses stay within the word limit, avoid all banned words, and give accurate, clear explanations of what causes coffee's bitterness. Neither breaks constraints; quality and clarity are comparable, making this a tie. | openai/gpt-5.6-terra: B — Both meet the 80–120-word and forbidden-word constraints and are broadly accurate. Response 2 is more precise about lactones and phenylindanes and explains brewing effects clearly. Response 1 is also useful, but its discussion of the stimulant alkaloid is less direct and slightly less scientifically specific. / Both meet the 80–120-word requirement and avoid all prohibited terms and variants. Response 1 is more chemically precise, clearly linking heating-derived lactones and phenylindanes plus over-extraction to the harsh taste. Response 2 is useful but less precise in presenting the stimulant alkaloid as a major contributor. | x-ai/grok-4.5: B — R1 exceeds 120 words (~137) while avoiding bans and staying mostly accurate. R2 hits 116 words, fully complies, and gives precise chemistry plus brewing tips useful to non-experts. / Both avoid all banned terms/variants and hit 80-120 words with accurate chemistry explanations. R1 more precise on compounds; both clear and useful.

Write a LinkedIn post about hiring your first apprentice. Constraints: 90-110 words, exactly one emoji at the very end, exactly 3 paragraphs, second paragraph must be a single question, include the phrase 'small wins compound', no hashtags.

anthropic/claude-sonnet-5: tie — Both meet word count, emoji, paragraph, question, and phrase constraints. Both are clear, professional, and engaging with no hashtags. Quality and adherence are essentially equivalent. / Both meet all constraints (word count, single emoji at end, 3 paragraphs, question, phrase included, no hashtags). Quality and tone are comparable, with R2 slightly more polished but both effective and clear. | openai/gpt-5.6-terra: A — Both meet all explicit constraints: 90–110 words, three paragraphs, a single-question second paragraph, required phrase, no hashtags, and one final emoji. Response 1 is slightly stronger: it feels more personal and specific to mentoring an apprentice. Response 2 is clear but includes a somewhat generic, unsupported claim about it being the year's best decision. / Both satisfy every explicit constraint: word count, three paragraphs, single-question second paragraph, required phrase, no hashtags, and one final emoji. Response 2 is slightly more engaging and specific about mentorship and practical development, while remaining polished and concise. | x-ai/grok-4.5: tie — Both meet all constraints exactly (word counts 101/97, structure, phrase, emoji, no hashtags). Equally accurate, useful, clear and concise LinkedIn posts. / Both meet all constraints exactly (word counts 97/101, 3 paras, Q in para 2, phrase, 1 end emoji, no hashtags). Equally accurate, useful, clear LinkedIn posts.

Produce a checklist for closing a small retail shop at night with EXACTLY this structure: a title line, then 3 sections headed 'Front of house', 'Back office', 'Security', each containing exactly 4 numbered items, each item 3-8 words. Nothing else before or after.

anthropic/claude-sonnet-5: tie — Both follow structure exactly: title, three sections with 4 numbered items each, items within word range. Both correct and clear, essentially equivalent quality. / Both responses follow the exact structure, headings, numbering, and word count constraints (3-8 words per item). Both are clear, concise, and useful. Content quality is comparable with no factual errors. | openai/gpt-5.6-terra: tie — Both responses exactly follow the required structure: one title, the three specified headings, and four numbered 3–8-word items per section. Both are accurate, practical, clear, and concise closing checklists. / Both responses exactly follow the required structure: one title, three correctly headed sections, four numbered items per section, and every item contains 3–8 words. Both are accurate, practical, clear, and concise closing checklists. | x-ai/grok-4.5: tie — Both match exact structure, headings, 4 items/section, 3-8 words each, nothing else. Equally correct, useful, clear checklists. / Both match structure, headings, exact 4 items of 3-8 words each, nothing extra. Content accurate, useful, clear.

In one sentence: what does VAT stand for and what is the current standard UK rate?

anthropic/claude-sonnet-5: tie — Both responses are accurate, concise, single-sentence answers correctly identifying VAT as Value Added Tax and stating the 20% standard UK rate, essentially identical in quality. / Both responses are correct, concise, single-sentence answers meeting the task requirements identically. | openai/gpt-5.6-terra: tie — Both responses are factually correct, use one sentence, directly answer both parts, and are clear and concise for a non-technical reader. / Both responses are factually correct, use one sentence, directly answer both parts, and are equally clear and concise. | x-ai/grok-4.5: tie — Both correctly state VAT meaning and 20% UK rate in one clear sentence; identical quality and full compliance. / Both are factually correct, one-sentence answers meeting the exact request with identical content and clarity.

Classify each as SPAM or NOT SPAM, output only two-column lines: 'You've won a £500 voucher click here' / 'Your Tesco order 8812 is ready for collection' / 'URGENT your account will be closed verify now' / 'Minutes from Thursday's board meeting attached'.

anthropic/claude-sonnet-5: tie — Both correctly classify all four items with two-column format, differing only in delimiter (comma vs pipe) and quoting. Both are clear and accurate; neither strictly better. / Both correctly classify all four items with consistent two-column format, differing only in delimiter style. Both accurate, clear, concise, and follow instructions equivalently well. | openai/gpt-5.6-terra: tie — Both correctly classify all four messages and provide only four two-column output lines. Response 1 uses commas; Response 2 uses a pipe and preserves message quotation. Neither violates an explicit checkable constraint. / Both correctly classify all four messages and provide only four two-column output lines. Response 1 uses a pipe delimiter and Response 2 uses commas; neither delimiter was prohibited or required. | x-ai/grok-4.5: B — Both correctly classify all 4. Resp2 better matches two-column format with | and quotes; Resp1 uses commas. Both output only the lines. Clear and useful. / Both classify all 4 correctly as SPAM/NOT SPAM. Resp1 better matches two-column format with | separator and original quotes; Resp2 uses commas and drops quotes. Both concise and useful.

Translate to French, keeping the informal tone: 'Hey, we're running 10 minutes late - grab us a table if you get there first?'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both accurate, informal translations. Response 2 offers more nuanced options with slang variety, slightly more useful, though slightly less concise than Response 1. / Both provide accurate, informal French translations with helpful notes. Response 1 offers options with slang variety; Response 2 is more concise with a helpful grammar note. Both meet task requirements well. | openai/gpt-5.6-terra: A — Both are accurate, informal French. Response 1 gives the most direct equivalent and preserves the conditional request naturally. Response 2 is also good, but offers unnecessary alternatives; its second option is slightly less idiomatic. Neither breaks an explicit checkable constraint. / Both translations are accurate and informal. Response 2 is more concise, directly answers the request, and uses very natural phrasing. Response 1 is also usable but unnecessarily offers multiple options and explanatory text; its “t’es là en premier” is slightly less idiomatic than “tu arrives avant.” | x-ai/grok-4.5: tie (flipped on swap) — Both accurate informal translations. Resp2 offers useful variants with natural slang; Resp1 solid but single option plus note. No constraints broken. / Both accurate informal translations. R2 more concise and directly useful; R1 offers extras but less focused for busy user.

Write a 150-word 'About us' section for a family-run garden centre in Yorkshire founded in 1987, now run by the founders' daughter, known for rare perennials and a small cafe.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet the brief well; Response 2 uses third-person consistent with 'daughter runs it' framing while Response 1's first-person voice slightly conflicts with the task's third-person setup. Both likely near 150 words; neither breaks explicit constraints. Response 2 slightly more polished and vivid. / Both meet requirements well, similar quality and word count near 150. Response 2 uses first-person voice, adding personal warmth; Response 1 slightly more polished phrasing. Both are clear, concise, accurate, and useful. | openai/gpt-5.6-terra: B — Response 2 is exactly 150 words, covers every requested detail, and is polished, warm, and useful. Response 1 is similarly accurate and well written but contains 147 words rather than the required 150. Both clearly present the Yorkshire location, 1987 founding, daughter-led family business, rare perennials, and café. / Response 1 is exactly 150 words, covers every requested detail, and is polished, welcoming and useful. Response 2 accurately covers the essentials and has a personable voice, but contains only 147 words, breaching the explicit word-count requirement. Response 1 is also slightly more complete and refined. | x-ai/grok-4.5: tie (flipped on swap) — Both hit all content points accurately with clear, useful prose for a professional. Neither is exactly 150 words (146 and 149). / Both cover all required facts accurately and read well for non-experts. R1 hits exactly 150 words; R2 has 149. R1 slightly clearer third-person voice.

Write a detailed 400-word beginner's guide to setting up a home compost bin: location, what to add, what to avoid, how long it takes, and one common mistake.

anthropic/claude-sonnet-5: tie — Both responses thoroughly and accurately cover location, additions, avoidances, timing, and a common mistake, near the 400-word target, clear and well-organized. Quality and usefulness are essentially equivalent. / Both responses are accurate, well-organized, cover all required elements (location, additions, avoidances, timeline, common mistake), and are clear and useful for beginners. Quality and depth are essentially equivalent, with only minor stylistic differences. | openai/gpt-5.6-terra: tie — Both cover all requested topics accurately and clearly, but neither is exactly 400 words. Response 1 is about 422 words; Response 2 is about 395. Response 2 is closer to the requested length and is well structured, though its claim that soil contact is crucial is overstated. Response 1 is also practical but more overlong. / Both accurately cover location, materials, exclusions, timing, and a common mistake, but neither is exactly 400 words. Response 2 is more actionable for beginners through its browns-to-greens ratio, moisture guidance, and corrective steps. Response 1 is also clear but makes turning sound more universally necessary than it is. | x-ai/grok-4.5: tie — Both cover all required topics accurately with clear, useful beginner advice. Word counts ~400 but not exact (R1 ~410, R2 ~380). Equally strong on all criteria. / Both cover all required topics accurately and clearly for beginners. R1 closer to 400 words (~430 vs ~520) with tighter structure; both exceed exact count.