{ "entity": "gpt-5-6-luna", "entity_type": "model", "category": "Spreadsheets", "suite": "suite-2026-08-tasks/spreadsheets", "model": "openai/gpt-5.6-luna", "judges": [ "anthropic/claude-sonnet-5", "google/gemini-3.1-pro-preview", "x-ai/grok-4.5" ], "judge_protocol": "judge-2026-08", "anchors": "solo-2026-08-a", "score": 98, "caps": 0, "cost_usd": 0.1139, "started_at": "2026-08-13T08:55:47.558Z", "finished_at": "2026-08-13T08:56:14.503Z", "tasks": [ { "id": "sp1", "name": "Excel formula from a description", "prompt": "In Excel, column A holds order dates and column B holds order values. Write a single formula for cell D1 that totals only the orders placed in the last full calendar month, relative to today's date. Give the formula and one sentence explaining it. Do not use a pivot table or a helper column.", "output": "```excel\n=SUMIFS(B:B,A:A,\">=\"&(EOMONTH(TODAY(),-2)+1),A:A,\"<\"&(EOMONTH(TODAY(),-1)+1))\n```\n\nThis sums order values whose dates fall from the first day through the end of the last completed calendar month.", "latency_ms": 4676, "cost": 0.0001909, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Formula correctly sums last full month using SUMIFS with EOMONTH boundaries, no helper column/pivot, concise explanation." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The formula is perfectly accurate and efficiently calculates the sum for the previous calendar month. The explanation is exactly one sentence, and all constraints are met." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Correct SUMIFS with EOMONTH for prior full month; exact format and one clear sentence met." } ] }, { "id": "sp2", "name": "Fix a broken formula", "prompt": "This Excel formula returns #VALUE! : =SUMIF(A:A,\">\"&TODAY()-30,B:B)+VLOOKUP(C1,Sheet2!A:D,5,FALSE). Identify every fault and give the corrected formula. Sheet2 has columns A to D only.", "output": "The fault is in the `VLOOKUP`:\n\n- `Sheet2!A:D` contains **four columns**, but the formula requests column **5**.\n- If the intended return column is D, the column index must be `4`.\n- The value returned by `VLOOKUP` must also be numeric because it is being added to the `SUMIF` result. If column D contains text, convert it to a number or correct the source data.\n\nAssuming the lookup value is in column A and the desired result is in column D, use:\n\n```excel\n=SUMIF(A:A,\">\"&TODAY()-30,B:B)+VLOOKUP(C1,Sheet2!A:D,4,FALSE)\n```\n\nIf you actually intended to return a fifth column, the range would need to include column E:\n\n```excel\n=SUMIF(A:A,\">\"&TODAY()-30,B:B)+VLOOKUP(C1,Sheet2!A:E,5,FALSE)\n```\n\nWith the original formula, the invalid column index normally produces `#REF!`; `#VALUE!` can occur if the matched value being added is text or an error value.", "latency_ms": 13250, "cost": 0.0007749, "task_score": 8, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 6, "violates": false, "reasoning": "Correctly identifies index/4 columns fault and fixes it, notes text-numeric issue; reasonably clear, though slightly hedges on #REF! vs #VALUE! cause without fully resolving it." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "Accurately identifies the column index error (which causes #REF!) and correctly explains that the #VALUE! error is caused by adding text to a number. Provides clear, corrected formulas." }, { "judge": "x-ai/grok-4.5", "score": 8, "violates": false, "reasoning": "Accurately spots VLOOKUP col-index error on 4-col range and notes #VALUE! causes; supplies correct formulas. Minor: singular 'fault' phrasing." } ] }, { "id": "sp3", "name": "Clean messy data", "prompt": "These postcodes were typed by hand: 'sw1a1aa', 'SW1A 1AA', ' m1 4wb', 'M14WB', 'ec1v9nr'. Give a single Excel formula that normalises any one of them to standard UK format (uppercase, one space before the final three characters). Assume the raw value is in A2.", "output": "```excel\n=LET(p,UPPER(SUBSTITUTE(A2,\" \",\"\")),LEFT(p,LEN(p)-3)&\" \"&RIGHT(p,3))\n```", "latency_ms": 5072, "cost": 0.0002298, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Formula correctly normalizes postcodes, uses valid Excel LET syntax; concise and correct, minor lack of explanation but meets task requirements well." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The formula perfectly normalizes the postcodes by removing all spaces, converting to uppercase, and inserting exactly one space before the last three characters. It is concise and accurate." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Formula correctly normalizes all examples to standard UK postcode format via uppercase, space removal, and reinsertion." } ] }, { "id": "sp4", "name": "Choose the right chart", "prompt": "A shop has 14 months of weekly revenue and wants to see whether sales are seasonal. In under 90 words, say which chart to build, which fields go on which axis, and one thing this chart will NOT tell them.", "output": "Build a **time-series line chart**. Put **week/date** on the x-axis and **weekly revenue** on the y-axis. This will reveal rises, dips, and possible repeating patterns over time. However, with only 14 months of data, it will **not reliably establish recurring seasonality**—there is barely more than one annual cycle to compare.", "latency_ms": 4751, "cost": 0.0001949, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Correct, concise, meets word limit, clear axes and limitation stated; could mention overlaying years for seasonality comparison for extra insight." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, stays well under the word limit, correctly identifies the chart and axes, and provides an insightful limitation regarding the 14-month timeframe." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Accurate chart advice, exact axes, valid limitation noted; under 90 words, fully addresses task clearly and usefully." } ] }, { "id": "sp5", "name": "Google Sheets QUERY", "prompt": "In Google Sheets, tab 'Sales' has headers in row 1: Date, Rep, Region, Amount. Write one QUERY() formula returning total Amount per Rep for the North region only, sorted highest first. Return only the formula.", "output": "=QUERY(Sales!A:D,\"select B, sum(D) where C = 'North' group by B order by sum(D) desc label sum(D) ''\",1)", "latency_ms": 1200, "cost": 0.00003, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Correct formula logic, proper syntax, sorted descending, filtered by North. Minor: label clearing could omit sum(D) header but not required; overall accurate and concise." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The formula perfectly matches the requirements, correctly referencing the columns, filtering, grouping, and sorting. It also strictly follows the constraint to return only the formula." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Correct QUERY formula matching all requirements exactly; returns only the formula as specified." } ] }, { "id": "sp6", "name": "Spot the error in the numbers", "prompt": "Monthly revenue: Jan 42,100; Feb 38,400; Mar 41,900; Apr 4,220; May 43,050; Jun 44,110. State in one sentence what is almost certainly wrong with this data and what the April figure most likely should be. Do not speculate about business causes.", "output": "April’s figure is almost certainly a data-entry error with a missing zero, and it most likely should be **42,200**.", "latency_ms": 2425, "cost": 0.00008, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 6, "violates": false, "reasoning": "Correctly identifies data error, gives plausible corrected value, but 42,200 is arbitrary guess not clearly justified; concise and follows one-sentence format reasonably." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, accurately identifies the missing zero typo, provides the correct likely figure, and uses exactly one sentence without business speculation." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Perfect one-sentence answer correctly identifying missing-zero error and 42,200 fix; fully follows all constraints with no speculation." } ] }, { "id": "sp7", "name": "Pivot logic in words", "prompt": "Explain to a non-technical manager, in exactly 3 bullet points of no more than 20 words each, what a pivot table does. Do not use the words 'aggregate', 'dimension' or 'axis'.", "output": "- Summarizes large amounts of data, helping you quickly see totals, averages, counts, and other key results.\n- Lets you rearrange information by categories, such as sales by region, product, month, or salesperson.\n- Makes comparisons and patterns easier to spot without changing the original data.", "latency_ms": 2904, "cost": 0.0001568, "task_score": 9.5, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Clear, accurate, non-technical, meets bullet count and word limits, avoids banned words." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all constraints. It provides exactly three bullet points, each under 20 words, avoids all forbidden terms, and explains pivot tables clearly and accurately for a non-technical manager." }, { "judge": "x-ai/grok-4.5", "score": null, "violates": null, "reasoning": "unparseable/failed" } ] }, { "id": "sp8", "name": "Percentage change trap", "prompt": "A price rose from £80 to £100, then fell from £100 to £80. State both percentage changes to one decimal place and explain in one sentence why they are not the same number.", "output": "Rise: **25.0%**. Fall: **20.0%**. They differ because each percentage is calculated using a different starting price (£80 versus £100).", "latency_ms": 3258, "cost": 0.0000772, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Correct percentages, one decimal place, clear one-sentence explanation, concise and accurate." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response accurately calculates both percentages to one decimal place and provides a clear, one-sentence explanation for the difference, following all instructions perfectly." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Correct percentages to 1dp; one-sentence explanation accurate and complete; fully meets all task requirements with clarity." } ] }, { "id": "sp9", "name": "Conditional formatting rule", "prompt": "In Excel, give the exact custom formula for a conditional formatting rule applied to A2:A500 that highlights a cell when its value is more than 20% below the average of that whole range. Return the formula and nothing else.", "output": "=A20,\"PAID\",\"UNPAID\")\n```\n\nIf the invoice number has a trailing space, it may not match the corresponding payment number and will be incorrectly reported as **“UNPAID.”**", "latency_ms": 2971, "cost": 0.0001242, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Correct formula and accurate explanation of trailing-space issue; concise and meets task requirements well." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The formula is correct and efficient. The explanation is exactly one sentence and accurately describes the issue with trailing spaces causing false 'UNPAID' results." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Correct COUNTIF formula; accurate one-sentence note on trailing-space mismatch causing false UNPAID." } ] }, { "id": "sp11", "name": "Explain a formula back", "prompt": "Explain in under 60 words what this does: =IFERROR(INDEX(B:B,MATCH(1,(C:C=\"UK\")*(D:D>1000),0)),\"none\"). Mention one reason it might be slow on a large sheet.", "output": "This formula returns the value in column B from the first row where column C equals “UK” and column D exceeds 1,000. If no match exists, it returns “none.” It may be slow on large sheets because it evaluates full-column ranges in an array calculation.", "latency_ms": 2295, "cost": 0.0000723, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Accurate, concise, meets word limit, explains slowness reason clearly. Minor: word count near 55, could be slightly clearer on array formula aspect." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly explains the formula and correctly identifies the performance issue with full-column array calculations, all well within the 60-word limit." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Accurate explanation of the array formula; under 60 words; correctly notes full-column array eval as slowdown cause; clear and complete." } ] }, { "id": "sp12", "name": "Refuse the impossible", "prompt": "Write an Excel formula that predicts next month's revenue with 100% accuracy from the previous 12 months. If this is not possible, say so plainly in one sentence and give the closest legitimate alternative instead.", "output": "No formula can predict next month’s revenue with 100% accuracy from only the previous 12 months; a legitimate alternative is `=FORECAST.ETS(EDATE(MAX(A2:A13),1),B2:B13,A2:A13)` (with dates in `A2:A13` and revenue in `B2:B13`).", "latency_ms": 4151, "cost": 0.0001787, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Correctly states impossibility, provides legitimate valid Excel forecasting formula, concise and clear, meets task requirements well." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, stating the impossibility in exactly one sentence and providing a highly accurate and useful alternative Excel formula." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Correctly states impossibility in one plain sentence and supplies a valid, well-specified FORECAST.ETS alternative." } ] } ] }