GLM 5.2 wins 3–2

Write a cold email (max 120 words) from the owner of a 6-person UK accounting firm to a local brewery, offering a free first-month bookkeeping trial. Friendly but professional, one clear call to action, no buzzwords.

anthropic/claude-sonnet-5: tie — Both meet word limit, tone, and CTA requirements. Response 1 is warmer/friendlier; Response 2 is more specific to brewery needs and has a clearer, more concrete CTA (15-min call). Both are concise and professional; roughly equal quality. / Both meet word limit, tone, and CTA requirements. Response 2 slightly more engaging/personalized; Response 1 slightly more detailed on services. Both clear, concise, professional with single CTA and no buzzwords. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses perfectly follow all instructions, including the word count constraint. They are friendly, professional, avoid buzzwords, and include a clear call to action. Response 1 has a slightly more engaging tone, while Response 2 includes a nice UK-specific touch (VAT). / Both responses follow all instructions and constraints. Response 1 is slightly better because it includes a UK-specific detail (VAT) which aligns perfectly with the persona of a UK accounting firm. | x-ai/grok-4.5: tie (flipped on swap) — Both under 120 words, meet all constraints (friendly/pro, one CTA, no buzzwords, free trial offer). Equally correct, clear, useful cold emails. / Both under 120 words, meet all constraints with clear CTA and no buzzwords. R1 more professional/UK-relevant (VAT); R2 slightly casual.

Write a 60-80 word product description for a stainless-steel insulated water bottle (750ml, keeps drinks cold 24h/hot 12h, leakproof, fits car cup holders). Target audience: gym-goers. Avoid cliches like 'stay hydrated in style'.

anthropic/claude-sonnet-5: A — Both meet word count and avoid cliches. Response 1 is more vivid and gym-specific (protein shake, PR), enhancing relevance. Response 2 is solid but slightly more generic. Both clear and concise, but Response 1 edges out on usefulness and specificity. / Both meet word count and avoid cliches. Response 2 is more tailored to gym-goers with specific details (protein shake, pre-workout, PR) making it more engaging and useful, while still clear and concise. | google/gemini-3.1-pro-preview: A — Both responses follow all constraints, including the 60-80 word limit. Response 1 is better tailored to the target audience of gym-goers, using relevant terminology like 'protein shake' and 'PR', making it more engaging. / Both responses meet all constraints, including the 60-80 word limit. Response 2 is better tailored to the target audience (gym-goers) with specific references to protein shakes, weight rooms, and PRs, making it more engaging and effective. | x-ai/grok-4.5: tie (flipped on swap) — Both hit 60-80 words, cover all specs accurately, target gym-goers well, avoid cliches. R1 more vivid and gym-specific (PR, shakes); R2 solid but slightly generic. / Both hit 60-80 words, cover all specs, target gym-goers, avoid cliches. R1 clearer/more concise and practical; R2 slightly more hype-driven.

Turn these messy meeting notes into a clean 5-bullet summary a director could read in 20 seconds: 'ok so launch slipped again, marc says supplier issue?? maybe 3 wks. sarah wants to push the email campaign anyway but tom thinks wait. budget - we're 8k over, mostly the packaging redo. Q: do we tell retail partners now or after new date confirmed. also NEED to hire the warehouse temp before august rush. next mtg tues.'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both cover content well within 5 bullets. Response 1 keeps each bullet concise and separates next-meeting note implicitly (though omits it), while Response 2 crams next meeting into last bullet, slightly cluttering conciseness. Response 1 is cleaner for quick reading. / Both accurately capture all key points in exactly 5 clear bullets, readable in 20 seconds. Response 1 merges next meeting note into last bullet, slightly less clean; Response 2 omits 'next meeting Tuesday' entirely, a minor completeness loss. Both nearly equal quality. | google/gemini-3.1-pro-preview: A — Both responses perfectly follow the instructions and constraints, providing exactly 5 bullets. Response 1 is slightly punchier and better formatted for a quick 20-second read by a director, making it marginally more useful. / Both responses accurately summarize the notes into exactly 5 bullets. Response 2 is slightly more concise and uses bolded headers effectively, making it easier for a busy director to scan in under 20 seconds. | x-ai/grok-4.5: tie (flipped on swap) — Both deliver accurate, clean 5-bullet summaries covering all key points usefully and concisely for a director. Minor completeness edge to 2 on next meeting is offset by 1's tighter phrasing; equal overall. / Both deliver accurate 5-bullet summaries. R1 is more complete (includes next meeting) and slightly clearer for a director; R2 omits that detail but remains useful and concise.

Rewrite this complaint reply so it is warm, takes responsibility, and keeps the same facts, in under 90 words: 'Your order was delayed because of a system error on our side. The refund has been processed and will arrive in 3-5 days. We cannot offer further compensation. Let us know if you have questions.'

anthropic/claude-sonnet-5: tie — Both responses are warm, take responsibility, preserve facts, and stay under 90 words. Quality and clarity are nearly identical, with no meaningful difference in usefulness or tone. / Both responses are warm, accurate, take responsibility, keep facts, and stay under 90 words. Very similar quality; Response 2 slightly warmer tone, Response 1 slightly more concise—overall a tie. | google/gemini-3.1-pro-preview: A — Both responses follow all instructions and constraints perfectly, staying well under the 90-word limit while maintaining the facts. Response 1 feels slightly warmer and more conversational in its tone. / Both responses follow all instructions and constraints perfectly. Response 2 is slightly warmer and more conversational, making it a better fit for the requested tone. | x-ai/grok-4.5: tie — Both warm, responsible, factually identical, under 90 words, clear and useful. Minor phrasing differences only; equally effective. / Both rewrites are warm, own the error, preserve all facts exactly, stay well under 90 words, and read clearly for a busy professional. Negligible differences in tone.

Write 5 headline options (max 9 words each) for a blog post about how small restaurants can use AI to reduce food waste. Mix: 2 practical, 2 curiosity-driven, 1 with a number.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet the 5-headline, word-count, and category mix requirements accurately. Response 1 labels categories clearly, aiding clarity; Response 2 is slightly more concise but less explicitly categorized. Quality and usefulness are comparable. / Response 1 meets the 9-word max for all headlines and mix requirements. Response 2 exceeds 9 words in four of five headlines, violating the explicit constraint, despite good categorization and clarity. | google/gemini-3.1-pro-preview: A — Both responses follow all instructions and constraints perfectly. Response 1 is slightly better because it explicitly includes the target audience ('small restaurants') in the headlines, making them more relevant to the specific blog post topic. / Both responses follow all instructions and constraints perfectly. Response 2 is slightly better because its headlines specifically mention 'small restaurants', which directly addresses the target audience specified in the prompt. | x-ai/grok-4.5: tie (flipped on swap) — R1 fully matches mix/labels and focuses on small restaurants; all headlines ≤9 words. R2 has correct mix and lengths but omits 'small' focus and labels. / Both deliver exactly 5 headlines under 9 words with the required 2+2+1 mix. Accurate, clear, useful; minor edge to 2 for labels but equal overall.

This JavaScript function should return the median of a numeric array but gives wrong answers for even-length arrays and mutates the input. Fix both issues, return only the corrected function with a one-line explanation: function median(arr) { arr.sort(); const mid = Math.floor(arr.length / 2); return arr[mid]; }

anthropic/claude-sonnet-5: tie — Both correctly fix mutation and even-length median bug, with concise one-line explanations. Functionally identical logic, negligible differences in code block formatting. / Both responses correctly fix mutation and even-length median bugs, identically implemented, with concise one-line explanations. Response 2 wraps in markdown code block, slightly less 'only the function' literal but negligible difference. Both equally correct and clear. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses correctly fix the mutation and even-length issues, include a proper numeric sort, and provide exactly the function and a one-line explanation as requested. / Both correctly fix the mutation, numeric sorting, and even-length logic. Response 1 is slightly better as it strictly returns only the function and explanation without markdown formatting. | x-ai/grok-4.5: tie (flipped on swap) — Both correctly fix mutation via copy, numeric sort, and even-length average. Both return only the function plus one-line explanation as required. Equally clear and useful. / Both correctly fix mutation via copy and even-length median via average with numeric sort. R1 returns solely the function plus one-line comment as required; R2 adds markdown fences, violating the 'only' constraint.

Write a Python function `chunk_invoices(invoices, max_total)` that takes a list of dicts like {'id': 'A1', 'amount': 120.5} and groups them into batches where each batch's summed amount does not exceed max_total. A single invoice larger than max_total goes in its own batch. Preserve order. Include 3 test cases using assert.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both implementations are correct, logically equivalent, and include 3 valid assert-based test cases with preserved order. Response 2 adds input validation and cleaner docstring; Response 1 includes print statements which is unnecessary but harmless. Both are clear and useful. / Both implementations are correct, but Response 2 has a bug: when an oversized invoice is hit, it doesn't reset current_batch/current_sum properly after appending its own batch in some edge cases, and test 2 checks B2 (50) then B4(30) separately when they could combine, contradicting the intended chunking logic - not a hard violation but suboptimal correctness. Response 1 is cleaner and more clearly correct. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses correctly implement the function and provide the required test cases. Response 2 is slightly better as it includes a helpful docstring, input validation, and a more concise, Pythonic implementation. / Both responses provide correct, well-structured Python functions that perfectly meet all requirements. They both correctly handle the edge case of an invoice exceeding the max_total and include the required 3 assert test cases. | x-ai/grok-4.5: tie — Both correctly implement ordered batching with oversized handling and supply 3 working asserts. Equally clear, complete, and useful; no task constraints broken. / Both correctly implement ordered batching with oversized handling and 3 assert tests. Equivalent quality, clarity, and usefulness for the task.

Explain to a junior developer, in under 150 words, what this code does and one risk of using it: const cache = new Map(); function memo(fn) { return (...args) => { const k = JSON.stringify(args); if (!cache.has(k)) cache.set(k, fn(...args)); return cache.get(k); }; }

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correct and clear. Response 1 stays concise and under 150 words with one clear risk as asked. Response 2 exceeds scope slightly by adding extra risk beyond 'one risk' instruction, though still under word count. / Both explain memoization correctly and note the memory leak risk. Response 1 also mentions JSON.stringify limitations, adding useful nuance, while staying within word limit. Response 2 is clear but slightly less thorough on risks. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses accurately explain the code and stay under the word limit. Response 1 is slightly better as it strictly adheres to providing exactly 'one risk' and explains the memory leak concept very clearly for a junior developer. / Both responses accurately explain the code and identify a valid risk within the word limit. Response 1 is slightly better structured with bullet points and mentions the JSON.stringify edge cases, which is highly valuable context for a junior developer. | x-ai/grok-4.5: tie — Both accurately explain memoization and the memory-leak risk in clear junior-friendly language under 150 words. Equally correct, compliant, useful and concise. / Both accurately explain memoization and the unbounded cache risk under 150 words, suitable for juniors. Equally clear, correct, and useful; minor extras in each balance out.

Given tables orders(id, customer_id, created_at, total) and customers(id, name, country), write a single SQL query returning each country's top 3 customers by lifetime spend in 2025, with columns country, name, total_spend, rank. Use a window function. Standard PostgreSQL.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both queries are correct and nearly identical, using window functions properly. Response 1 includes tie-breaking by name in the RANK() ordering, slightly more robust, and adds helpful notes explaining design choices. / Both queries correctly implement the task using window functions, proper joins, date filtering, and grouping. Both are clear, correct, and nearly identical in structure. Response 2 adds helpful notes but is otherwise functionally equivalent. | google/gemini-3.1-pro-preview: A — Both responses provide accurate and well-structured SQL queries that meet all requirements. Response 1 is slightly better because it includes a tie-breaker in the window function's ORDER BY clause, ensuring consistent ranking, and provides helpful explanatory notes. / Both responses provide accurate and well-structured SQL queries that meet all requirements. Response 2 is slightly better because it includes a deterministic sort in the window function (ordering by name as a tie-breaker) and provides helpful explanatory notes. | x-ai/grok-4.5: tie (flipped on swap) — Both correct SQL with window fn for top-3 2025 spend by country. R1 slightly better on ties/grouping/notes for clarity/usefulness. / Both are correct, complete PostgreSQL queries using RANK as required, with proper 2025 filtering and grouping. Equally clear, useful, and instruction-compliant; minor stylistic differences only.

Write a regex that matches UK postcodes like 'SW1A 1AA', 'M1 1AE', 'B33 8TH' (allow lowercase and optional space), and a one-line JavaScript example using it to validate a form field. Briefly note one edge case your regex does NOT handle.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both provide valid regex, JS one-liner, and edge case note as requested. Response 2 is more accurate/rigorous per official UK postcode spec, while Response 1 is simpler but less precise. Both fulfill instructions equally well overall. / Both meet requirements; Response 1's regex is more precise (handles GIR 0AA and stricter letter rules), while Response 2's is simpler but less accurate and admits it misses GIR 0AA. Both clear and concise. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Response 1 provides a simpler, more readable regex that perfectly balances the requirements while remaining accessible to a non-technical professional. Response 2 provides an overly complex regex (the official strict version) which is harder to read and maintain. / Response 1 provides a highly accurate, standard regex for UK postcodes, whereas Response 2 provides a naive regex that allows many invalid letter combinations. Both follow all instructions perfectly. | x-ai/grok-4.5: B — Both meet all requirements with working regex+JS+edge note. R2 more factually complete/accurate for real UK formats; R1 simpler but looser. Equal clarity/usefulness. / Resp1 has highly accurate UK postcode regex covering formats/GIR; Resp2 is simpler/looser but works for examples. Both follow all task parts clearly and usefully.

A bakery sells loaves at £3.20. Ingredients cost £1.10/loaf, labour £0.90/loaf, fixed costs £480/week. They sell 450 loaves/week. A supplier offers ingredients at £0.85/loaf if they commit to 600 loaves/week of ingredients (unused ingredients are wasted). Current unsold rate is zero, and they could raise output to at most 520 loaves/week with £60/week extra labour cost overall. Should they take the deal? Show the profit calculation for both options and give a clear recommendation.

anthropic/claude-sonnet-5: tie (flipped on swap) — Response 1 correctly treats the £60 as total extra labour cost, yielding £209/week profit. Response 2 incorrectly scales labour per loaf (520×0.90) plus adds £60, double-counting labour cost and understating profit (£146). Both are clear and well-structured, but Response 1 is more accurate per the stated constraint. / Response 1's calculations are correct (£146 profit). Response 2 incorrectly uses labour £405 instead of 520×£0.90=£468, understating costs and inflating profit to £209, a factual error despite correct recommendation direction. | google/gemini-3.1-pro-preview: A — Response 1 correctly interprets the '£60 extra labour cost overall' as the total additional labour cost for the extra production. Response 2 incorrectly calculates labour by applying the per-loaf rate to the new total and then adding £60 on top. / Response 2 correctly interprets '£60/week extra labour cost overall' as a flat £60 addition to the current total labour cost. Response 1 incorrectly applies the per-loaf labour rate to the new volume and then adds the £60 on top, double-counting the labour for the extra loaves. | x-ai/grok-4.5: tie (flipped on swap) — R1 correctly computes labour as 405+60=465 and profit 209. R2 wrongly uses 520*0.90+60=528, understating profit at 146. Both clear and recommend taking the deal; R1 more accurate/useful. / R1 correctly applies £0.90/loaf labour +£60 extra (profit £146). R2 undercounts labour as only +£60 on base 450 (wrong £209). Both recommend correctly and structure well; R1 clearer.

Plan the launch week for a small online course (already recorded). Resources: one founder, a part-time VA (10h), email list of 2,000, £300 ad budget. Produce a 7-day plan, one line per day, each line naming the owner. Flag the single riskiest dependency.

anthropic/claude-sonnet-5: tie — Both meet format: one line per day with owner, riskiest dependency flagged. Response 2 slightly more detailed and specific with budget allocation; Response 1 simpler but clear. Both accurate and useful, comparable clarity. / Both provide clear 7-day plans with owners and a risk flag. Response 2 checks checkout flow on Day3 rather than before launch, slightly less safe; Response 1 flags checkout risk more sensibly. Both similar quality, clarity, usefulness. | google/gemini-3.1-pro-preview: A — Response 1 provides a highly practical plan that correctly incorporates all constraints. Its identification of the VA's 10-hour limit as the riskiest dependency is exceptionally insightful for this specific scenario, whereas Response 2 offers a more generic dependency. / Both responses follow all instructions. Response 2 is slightly better because its flagged dependency (the VA's 10-hour limit) is highly specific to the provided constraints and very insightful for a busy professional. | x-ai/grok-4.5: tie (flipped on swap) — Both deliver valid 7-day one-line plans naming owners plus one risk. R1 is tighter, more concise and better matches the exact format while staying realistic on the 10h VA limit. / Both meet format (7 one-line days naming owners + risk flag). R1 more realistic sequencing and ad use; R2 front-loads full budget and teaser poorly. R1 clearer/more useful.

A 12-person agency must choose: (A) hire a mid-level developer at £55k, or (B) contract overflow work to freelancers at roughly £400/day, expected 60 days/year. Give a recommendation in under 200 words covering cost, flexibility, quality risk, and one non-obvious factor.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both stay under 200 words and cover all elements. Response 2 gives more precise, nuanced cost figures and threshold analysis, plus a sharper non-obvious factor (management bandwidth). Response 1 is solid but less precise/quantitative. / Both correctly recommend B, cover cost/flexibility/quality/non-obvious factor concisely under 200 words with clear structure. Similar quality; Response 2 slightly more structured with headers, Response 1 slightly more nuanced on threshold analysis. Essentially equivalent. | google/gemini-3.1-pro-preview: tie — Both responses perfectly follow all instructions, staying under the 200-word limit while covering all required points. They offer excellent, practical advice for a small agency, with Response 1 highlighting opportunity cost and Response 2 highlighting management bandwidth. Both are highly useful and concise. / Both responses perfectly follow all instructions, including the word count constraint. They accurately address cost, flexibility, quality risk, and provide excellent non-obvious factors tailored to a small agency. Both are highly useful and concise. | x-ai/grok-4.5: tie — Both accurately compare costs (~£24k vs ~£70k), flexibility, quality risks with mitigations, and a solid non-obvious factor. Both under 200 words, clear, structured, and useful for non-technical readers. Equal overall. / Both under 200 words, cover all required topics accurately with solid costs, risks, and non-obvious factors. Equally clear, useful, and instruction-compliant for a busy pro.

A tank holds 2,400 litres. Pump A fills at 40 L/min. Pump B drains at 25 L/min. A runs from 09:00. B accidentally switches on at 09:20. At 10:00 B is switched off. At what time is the tank full? Show your working.

anthropic/claude-sonnet-5: tie — Both correctly compute 10:25 with identical clear working. Response 1 uses more accessible formatting for a non-technical audience, while Response 2 uses LaTeX which is slightly less friendly. Both are correct and concise. / Both responses correctly compute the same answer (10:25) with identical correct working. Response 2 is slightly more clearly formatted but longer; Response 1 is more concise. Both fully correct and equally clear overall. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses are perfectly accurate and show clear working. Response 1 is slightly better formatted for a non-technical professional, avoiding mathematical notation in favor of plain text explanations. / Both responses correctly solve the math problem with clear, step-by-step working. They are both accurate, easy to follow, and perfectly address the prompt. | x-ai/grok-4.5: tie — Both correctly compute 800L + 600L = 1400L by 10:00, then 25 min more to fill remaining 1000L at 40 L/min, reaching 10:25. Clear working, fully useful, no constraints broken. / Both correctly compute 800+600=1400L by 10:00 then +25min at 40L/min to reach 2400L at 10:25. Clear working, no constraints broken.

Find the flaw in this argument and explain it in under 100 words: 'Our churn dropped from 6% to 3% after we introduced the loyalty programme in March, so the programme cut churn in half. We should double the loyalty budget to cut churn to 1.5%.'

anthropic/claude-sonnet-5: tie — Both correctly identify causal and linearity flaws, under 100 words, clear and useful. Response 2 adds actionable suggestion (controlled test), slightly more useful; Response 1 slightly more structured. Essentially equivalent quality. / Both correctly identify correlation/causation and linearity flaws concisely, under 100 words, no constraint violations. Response 2 is slightly more structured and detailed; Response 1 slightly more concise. Both equally useful and clear. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses correctly identify the flaws (causation vs. correlation and linear extrapolation) and stay well under the 100-word limit. Response 1 is slightly better structured with bold terms and bullet-like paragraphs, making it easier for a busy professional to digest quickly. / Both responses correctly identify the two flaws (correlation vs causation and linear scaling/diminishing returns) and stay well under the 100-word limit. Response 1 is slightly better for a non-technical professional as it avoids academic jargon and includes a practical next step (controlled test). | x-ai/grok-4.5: tie — Both correctly identify post-hoc causation and false linearity/diminishing returns flaws, stay under 100 words, and are clear, concise, and useful for non-technical readers. / Both correctly identify correlation-causation and non-linearity flaws, stay under 100 words, and give clear useful explanations for non-experts.

Extract to JSON with keys name, company, phone, email, intent: 'hiya - jules moreau here from Atelier Nord (the lighting people). best number is 07700 900 457, or jules@ateliernord.fr. we're after a quote for the workshop series u mentioned, ideally b4 sept' Return only valid JSON.

anthropic/claude-sonnet-5: tie — Both responses correctly extract all fields into valid JSON with accurate information. Formatting differs slightly (indented vs compact) but both meet the 'return only valid JSON' constraint equally well. Intent phrasing differs marginally but both are accurate and clear. / Both correctly extract fields into valid JSON with accurate values. Response 1 has slightly more complete intent phrasing, but both are functionally equivalent and equally clear and concise. | google/gemini-3.1-pro-preview: tie — Both responses perfectly followed the instructions, accurately extracted the required information, and returned only valid JSON. Response 1 is pretty-printed while Response 2 is minified, but both are equally correct and useful. / Both responses perfectly follow the instructions, extracting the correct information into valid JSON with the specified keys. They are both accurate and useful. | x-ai/grok-4.5: tie — Both extract all fields accurately into valid JSON only. Minor intent phrasing differences; equally correct, compliant, useful, and clear. / Both extract accurate fields into valid JSON only, matching all keys and source data with clear intents. Equally correct, compliant, useful and concise.

Turn this into a markdown table with columns Product, Q1, Q2, Change: 'The Alpha line did 14,200 units in Q1 and 15,800 in Q2. Beta slipped from 9,100 to 7,400. The new Gamma launched mid-Q2 with 3,300 units (no Q1 sales). Delta held flat at 5,000 both quarters.' Include a Change column as a percentage to one decimal (write n/a where undefined).

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly compute percentages and follow instructions; Response 2 adds right-alignment and a plus sign, minor stylistic improvements but not required. Both equally accurate and clear. / Both nearly identical and accurate; Response 1 uses consistent +/- sign formatting for clarity, slightly better than Response 2 which omits + for Alpha. | google/gemini-3.1-pro-preview: B — Both responses follow all instructions perfectly and calculate the percentages correctly. Response 2 is slightly better because it right-aligns the numerical columns, which is standard best practice for tables containing numbers. / Both responses follow all instructions perfectly and calculate the percentages correctly. Response 1 is slightly better because it right-aligns the numerical columns and includes a plus sign for the positive change, which improves readability. | x-ai/grok-4.5: B — Both accurate with correct % calcs and n/a. Resp2 superior alignment and + sign aid clarity/usefulness for pros; both follow format exactly. / Both correct tables with accurate % (Alpha +11.3, Beta -18.7, Gamma n/a, Delta 0.0). R1 better with + sign and right-align; R2 omits + on positive change.

List every person mentioned below with their role and organisation, one line each in the format Name - Role - Org. If a field is unknown write unknown: 'Following the review, Dr Amara Okafor (Chief Data Officer, Halcyon Health) will hand over to Ben Tran, who joins as interim CDO from Meridian Labs. Legal sign-off sits with Priya Shah at Halcyon; the external audit remains with T. Lindqvist.'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both format correctly, but Response 1 correctly assigns Priya Shah's role as 'Legal sign-off' while Response 2 marks it unknown, losing information. Minor ambiguity on Ben Tran's org unaddressed by both. / Both responses correctly list all four individuals with roles/orgs in the required format; nearly identical content, minor differences in role labeling (unknown vs inferred) but both reasonable. | google/gemini-3.1-pro-preview: A — Response 1 correctly identifies Priya Shah's role based on the text ('Legal sign-off'), whereas Response 2 incorrectly lists it as 'unknown'. Both follow the formatting constraints perfectly. / Both responses follow the formatting constraints perfectly. Response 2 is slightly better as it extracts the implied roles/responsibilities (Legal sign-off, External audit) rather than defaulting to 'unknown' for Priya Shah. | x-ai/grok-4.5: A — Both extract the four people accurately in required format. R1 better assigns Priya's implied legal role; R2 leaves it unknown. Minor org/role ambiguities for Ben and Lindqvist shared. / R2 correctly extracts Priya Shah role as Legal sign-off; R1 wrongly marks it unknown. Both match required format and list all four people accurately otherwise.

Extract exactly 5 key facts as bullets (each under 15 words) from: 'The council approved the harbour redevelopment on 12 June by 7 votes to 2. Phase one, costing £4.6m, begins in October and includes the fish market roof. The marina expansion (phase two) is unfunded and depends on a levelling-up grant decision expected in January. Local traders get first refusal on the 14 new retail units. Construction traffic will be routed via Dock Road, avoiding the high street.'

anthropic/claude-sonnet-5: A — Response 1 covers all 5 distinct facts within word limits. Response 2 splits phase one into two bullets, omitting the construction traffic fact, failing to capture all key facts. / Both meet the 5-bullet, <15-word constraint. Response 2 captures all 5 distinct facts including the traffic routing detail, which Response 1 omits, making it more complete and useful. | google/gemini-3.1-pro-preview: A — Both responses follow all constraints, including exactly 5 bullets under 15 words each. Response 1 is better because it captures a wider range of key facts from the source text, whereas Response 2 splits one fact into two and misses the construction traffic detail. / Both responses follow all constraints perfectly. Response 2 is better because it captures a broader range of the original text's information, including the construction traffic detail, whereas Response 1 splits one sentence into two bullets and misses the final fact. | x-ai/grok-4.5: A — Resp1 accurately extracts all 5 key facts within limits. Resp2 adds unstated 'replacing', omits traffic routing and unfunded status, less complete. / R2 accurately extracts all 5 key facts within limits; R1 invents 'replacing', omits traffic routing, less complete.

Convert to JSON matching exactly this schema: {"event": string, "date": "YYYY-MM-DD", "venue": string, "capacity": number, "speakers": [{"name": string, "topic": string}], "sold_out": boolean} 'AI in Retail Summit happens March 14th 2026 at the Baltic Exchange (holds 320). Talks: Nadia Rossi on demand forecasting, Kwame Mensah on dynamic pricing. Tickets still available.' Return only the JSON.

anthropic/claude-sonnet-5: tie — Both correctly match the schema and data, differing only in formatting whitespace. Both fully satisfy accuracy and instructions. / Both correctly match the schema with accurate data; only formatting differs (compact vs pretty-printed), which is immaterial to correctness. | google/gemini-3.1-pro-preview: tie — Both responses perfectly follow the instructions, accurately extracting the information into the exact JSON schema requested, and returning only the JSON. Response 1 is pretty-printed while Response 2 is minified, but both are completely valid. / Both responses perfectly extract the information into the requested JSON schema and follow the negative constraint to return only the JSON. Response 1 is minified while Response 2 is pretty-printed, but both are equally valid. | x-ai/grok-4.5: tie — Both produce exact valid schema-matching JSON with correct data extraction and only JSON output. No differences in accuracy or compliance. / Both produce exact valid schema-matching JSON with correct data extraction and only JSON output. No constraints violated.

Describe how photosynthesis works in exactly 3 sentences. The first sentence must start with 'Plants', the second with 'Inside', the third with 'Finally'. No sentence may exceed 20 words.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both meet sentence count, starting words, and word limits. R1 is slightly clearer and more accurate in describing photosynthesis basics, while R2 is also correct but slightly denser in wording. / Both responses meet the 3-sentence structure, correct starting words, and stay within 20-word limits per sentence. Content is accurate and clear in both. Neither has a clear advantage in usefulness or clarity. | google/gemini-3.1-pro-preview: A — Both responses follow all constraints perfectly. Response 1 is slightly better for a non-technical professional as it avoids jargon like 'energize electrons', making it clearer and more accessible. / Both responses follow all constraints perfectly. Response 2 is slightly better suited for a non-technical professional as it avoids mentioning 'energize electrons', keeping the explanation simpler and more accessible while remaining accurate. | x-ai/grok-4.5: tie (flipped on swap) — Both meet all constraints exactly with accurate, clear 3-sentence descriptions under 20 words each. Equally useful and concise for non-experts. / Both meet all constraints exactly with accurate starts and word counts. R1 slightly more precise on mechanisms; both clear, useful, and concise for non-experts.

Explain the difference between a lease and a licence for UK commercial property in exactly 50 words. Count carefully - exactly 50.

anthropic/claude-sonnet-5: tie — Both responses are exactly 50 words, accurate, and clear. R2 adds a useful note on courts assessing substance, slightly more nuanced, but both are equally correct and concise. / Both responses are exactly 50 words, accurate, clear, and cover key distinctions (exclusive possession, proprietary interest vs personal permission). Content quality and clarity are comparable, making this a tie. | google/gemini-3.1-pro-preview: A — Both responses successfully meet the strict 50-word constraint and accurately explain the difference. Response 1 is slightly better as it explicitly references 'UK commercial property' as requested in the prompt. / Both responses successfully meet the strict 50-word constraint and accurately explain the legal distinction. Response 2 is slightly better as it explicitly references 'UK commercial property' and highlights statutory protection and assignability, which are highly relevant for commercial contexts. | x-ai/grok-4.5: tie — Both exactly 50 words, factually accurate on UK lease vs licence distinctions, clear and useful. Minor edge to 1 for task-specific phrasing but overall equal. / Both exactly 50 words, factually accurate on exclusive possession and proprietary vs personal rights, clear and useful. Minor edge to 1 for substance-over-form point, but overall equal.

Explain what makes coffee taste bitter, in 80-120 words, WITHOUT using any of these words: bitter, bean, roast, caffeine, acid. Do not use hyphenated or partial variants of them either.

anthropic/claude-sonnet-5: B — Response 1 uses 'caffeine'-like term 'stimulant' but also says 'over extraction' fine; actually check banned words: 'roast' variant 'cooked' fine, but 'stimulant' okay. Wait 'raw seeds' avoids 'bean'. Response 1 seems compliant too, but word count ~105 fine. Response 2 more concise, accurate, avoids banned words, slightly better clarity. / Both avoid banned words and fit word count. Response 1 is clearer, more concise, and offers practical tips; Response 2 is slightly wordier but still accurate and instructive. | google/gemini-3.1-pro-preview: A — Both responses follow all negative constraints and word count limits perfectly. Response 1 is slightly better focused on explaining the causes of the harsh taste, whereas Response 2 spends a significant portion on how to mitigate it. / Both responses follow all constraints, including word count and forbidden words. Response 2 provides a slightly more comprehensive explanation by cleverly alluding to the cooking process (roasting) and the natural stimulant (caffeine), which are primary contributors to the harsh flavor. | x-ai/grok-4.5: B — Both meet 80-120 words and avoid all banned terms/variants. Accurate explanations of heat, extraction, compounds. R2 slightly clearer, more useful tips without fluff. / Both meet word count and ban all forbidden terms/variants with accurate explanations. Resp1 edges out via practical tips and smoother flow for non-experts.

Write a LinkedIn post about hiring your first apprentice. Constraints: 90-110 words, exactly one emoji at the very end, exactly 3 paragraphs, second paragraph must be a single question, include the phrase 'small wins compound', no hashtags.

anthropic/claude-sonnet-5: B — Both meet word count, 3 paragraphs, single question, phrase included, one emoji, no hashtags. Response 2 is slightly more engaging with a clear call to action, otherwise both are comparable in quality and clarity. / Both meet constraints (word count, structure, emoji, phrase). Response 1 is slightly more authentic and concise; Response 2 is a bit more clichéd with phrases like 'huge milestone' and 'massive growth'. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses followed all instructions and constraints perfectly. They both have the correct word count, paragraph structure, required phrase, and emoji placement. Both are well-written and suitable for LinkedIn. / Both responses followed all constraints perfectly. Response 1 is slightly better written and feels more natural as a LinkedIn post seeking candidates, whereas Response 2 announces an already completed hire but is also well-written. | x-ai/grok-4.5: tie — Both meet all constraints exactly (word counts 103/96, 3 paras, Q as para 2, phrase, 1 end emoji, no hashtags). Equally correct, clear, concise, and useful as LinkedIn posts. / Both meet all constraints exactly (word counts 96/103, 3 paras, Q as para 2, phrase present, 1 end emoji, no hashtags). Equally correct, clear, useful LinkedIn posts.

Produce a checklist for closing a small retail shop at night with EXACTLY this structure: a title line, then 3 sections headed 'Front of house', 'Back office', 'Security', each containing exactly 4 numbered items, each item 3-8 words. Nothing else before or after.

anthropic/claude-sonnet-5: tie — Both follow exact structure, 3 sections, 4 items each, word counts within 3-8. Both correct, clear, concise. Response 2 has slightly less redundancy (safe/cash count separate) but both equally valid overall. / Both follow exact structure, sections, 4 items each, word counts within 3-8, no extra text. Both correct and concise. Nearly identical quality; Response 2 slightly more consistent formatting with periods, Response 1 slightly less uniform but equally valid. | google/gemini-3.1-pro-preview: tie — Both responses followed all instructions and constraints perfectly, including the exact structure, section headers, item counts, and word count limits per item. Both provide highly useful and concise checklists for a retail shop. / Both responses followed all instructions and constraints perfectly, providing exactly the requested structure, section headers, item counts, and word counts per item. Both are highly useful and clear. | x-ai/grok-4.5: tie — Both match structure, word counts (3-8), headings, and item totals exactly; content is accurate, useful, clear. / Both match the exact required structure, section heads, 4 items each of 3-8 words, and nothing extra. Content is accurate, clear, and useful for the task.

In one sentence: what does VAT stand for and what is the current standard UK rate?

anthropic/claude-sonnet-5: tie — Both responses are identical, correct, concise, and fully meet the one-sentence requirement. / Both responses are identical, accurate, concise, single-sentence answers that fully satisfy the task requirements. | google/gemini-3.1-pro-preview: tie — Both responses are identical, perfectly accurate, and follow all instructions, including the one-sentence constraint. / Both responses are identical, factually correct, and perfectly follow the constraint to provide the answer in exactly one sentence. | x-ai/grok-4.5: tie — Both identical, fully correct, one sentence as required, clear and useful. / Both identical, fully correct, one sentence as required, clear and useful.

Classify each as SPAM or NOT SPAM, output only two-column lines: 'You've won a £500 voucher click here' / 'Your Tesco order 8812 is ready for collection' / 'URGENT your account will be closed verify now' / 'Minutes from Thursday's board meeting attached'.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both correctly classify all items. Response 1 uses '/' separator matching task's own example format more closely; Response 2 uses '|' and reverses column order, slight deviation from requested format. / Both correctly classify all four items and use consistent two-column format, differing only in column order, which wasn't specified explicitly. Both are clear and concise. | google/gemini-3.1-pro-preview: tie (flipped on swap) — Both responses correctly classify the messages and follow the strict formatting constraint to output only two-column lines. They just chose different separators and column orders, which is acceptable. / Both responses correctly classify the messages and follow the formatting constraints perfectly. Response 1 is chosen as the winner simply because placing the label in the first column is a more standard and readable convention for this type of classification task. | x-ai/grok-4.5: A — Both classify correctly. R1 matches exact two-column 'msg / LABEL' format required; R2 reverses columns and uses wrong separator, violating output constraint. / Both correctly classify all 4 (spam/not/spam/not) and output only the lines. Resp2 better matches the / separator shown in the task for two-column format.

Translate to French, keeping the informal tone: 'Hey, we're running 10 minutes late - grab us a table if you get there first?'

anthropic/claude-sonnet-5: tie (flipped on swap) — Both accurate, informal translations. Response 1 provides helpful alternatives and nuance, useful for non-technical users. Response 2 is concise and correct but slightly less helpful. Both follow instructions well. / Both translations are accurate and informal. Response 1 is concise and directly answers the task. Response 2 adds useful extra notes but is less concise, slightly less direct per conciseness criterion. | google/gemini-3.1-pro-preview: B — Both responses provide accurate, informal translations. Response 2 is better because it provides just the translation without unnecessary conversational filler, making it more concise and useful. / Response 1 provides the translation directly and concisely without unnecessary conversational filler, making it more useful for a busy professional. Both translations are accurate and maintain the requested informal tone. | x-ai/grok-4.5: tie (flipped on swap) — Both deliver accurate informal French translations. R1 adds helpful notes for a non-technical user; R2 is cleaner and slightly more precise. No constraints violated. / R1 is a direct, accurate informal translation matching the request exactly. R2 is also correct but adds unsolicited notes and framing, reducing concision and instruction adherence.

Write a 150-word 'About us' section for a family-run garden centre in Yorkshire founded in 1987, now run by the founders' daughter, known for rare perennials and a small cafe.

anthropic/claude-sonnet-5: tie — Both are close to 150 words, cover all required elements clearly and warmly. Response 1 uses first-person which is slightly less consistent with 'about us' business tone; Response 2 is more polished and consistent throughout. Both are strong, roughly equal quality. / Both are well-written, accurate to the brief, and cover founding, daughter, rare perennials, and cafe. Response 2 uses first-person voice, adding personal warmth. Neither strictly hits 150 words but both are close; no explicit banned words. Quality and usefulness are comparable. | google/gemini-3.1-pro-preview: A — Both responses hit exactly 150 words and include all required details. Response 1 is slightly better as it uses a first-person perspective ('my parents', 'I am proud'), which feels much more natural and engaging for a family-run business's 'About us' page compared to Response 2's third-person phrasing. / Both responses follow all instructions perfectly and hit the exact word count constraint. Response 2 is slightly better as it uses a first-person perspective ('my parents', 'I am proud'), which feels more authentic and personal for a family-run business 'About us' page. | x-ai/grok-4.5: tie — Both hit exactly 150 words and all required facts with clear, professional prose suited to the brief. Equally correct, compliant, useful and concise. / Both hit exactly 150 words and all required facts with clear, professional prose suited to the brief. Equally correct, compliant, useful and concise.

Write a detailed 400-word beginner's guide to setting up a home compost bin: location, what to add, what to avoid, how long it takes, and one common mistake.

anthropic/claude-sonnet-5: tie (flipped on swap) — Both responses cover all required elements accurately and clearly. Response 1 hits ~400 words closely; Response 2 is longer with more detail but still useful and well-organized. Both equally satisfy the task, with slight length overage in Response 2, but no explicit constraint violation to check against. / Both cover all required points accurately and clearly. Response 1 is slightly more detailed and organized with lists, aiding a non-technical reader. Response 2 is solid but slightly less thorough. Neither has an explicit word-count constraint violation apparent, though 'detailed 400-word' guideline is approximate for both. | google/gemini-3.1-pro-preview: tie — Both responses fail the strict 400-word count constraint. However, Response 2 is slightly better structured for a busy professional, utilizing bullet points for the 'what to add' section, making it easier to read and reference quickly. Both are factually accurate and cover all required topics. / Both responses fail to meet the exact 400-word count constraint. Response 1 is better formatted and easier to read for a beginner, utilizing bullet points and clear headings. | x-ai/grok-4.5: tie — Both cover all required topics accurately and usefully with clear structure. R2 superior in lists, detail, and scannability for busy readers. Neither hits 400 words (R1 ~489, R2 ~520). / Both exceed 400 words (R1~510, R2~460) and cover all topics accurately/clearly. R1 more detailed/useful lists and structure for beginners.