{ "entity": "gpt-5-3-codex", "entity_type": "model", "category": "Spreadsheets", "suite": "suite-2026-08-tasks/spreadsheets", "model": "openai/gpt-5.3-codex", "judges": [ "anthropic/claude-sonnet-5", "google/gemini-3.1-pro-preview", "x-ai/grok-4.5" ], "judge_protocol": "judge-2026-08", "anchors": "solo-2026-08-a", "score": 98, "caps": 0, "cost_usd": 0.1745, "started_at": "2026-08-13T08:54:49.757Z", "finished_at": "2026-08-13T08:55:15.270Z", "tasks": [ { "id": "sp1", "name": "Excel formula from a description", "prompt": "In Excel, column A holds order dates and column B holds order values. Write a single formula for cell D1 that totals only the orders placed in the last full calendar month, relative to today's date. Give the formula and one sentence explaining it. Do not use a pivot table or a helper column.", "output": "```excel\n=SUMIFS(B:B, A:A, \">=\"&EOMONTH(TODAY(),-2)+1, A:A, \"<\"&EOMONTH(TODAY(),-1)+1)\n```\n\nThis sums values in column B where the date in column A is from the first day of last month through the last day of last month (using `<` the first day of this month to safely include any time stamps).", "latency_ms": 3165, "cost": 0.00391125, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Correct, single formula, no pivot/helper column, clear concise explanation matching constraints." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response provides a perfectly accurate formula that elegantly handles time stamps, follows all constraints including the one-sentence explanation, and is highly useful and concise." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Correct SUMIFS with EOMONTH bounds for prior full month; exact one-sentence explanation; meets all constraints." } ] }, { "id": "sp2", "name": "Fix a broken formula", "prompt": "This Excel formula returns #VALUE! : =SUMIF(A:A,\">\"&TODAY()-30,B:B)+VLOOKUP(C1,Sheet2!A:D,5,FALSE). Identify every fault and give the corrected formula. Sheet2 has columns A to D only.", "output": "You have **one definite formula fault** and one likely logic issue:\n\n1. **`VLOOKUP` column index is invalid** \n - `Sheet2!A:D` has only **4 columns**, but you asked for column **5**. \n - This must be `4` (or expand the lookup range).\n\n2. *(Logic, not syntax)* Your `SUMIF` criteria `\">\"&TODAY()-30` includes **everything after 30 days ago**, including future dates. \n - If you truly mean “last 30 days”, use `SUMIFS` with an upper bound too.\n\n### Minimal corrected version (fixes the hard error):\n```excel\n=SUMIF(A:A,\">\"&TODAY()-30,B:B)+VLOOKUP(C1,Sheet2!A:D,4,FALSE)\n```\n\n### Better “last 30 days” version:\n```excel\n=SUMIFS(B:B,A:A,\">=\"&TODAY()-30,A:A,\"<=\"&TODAY())+VLOOKUP(C1,Sheet2!A:D,4,FALSE)\n```\n\n(Optional robustness: wrap `VLOOKUP` in `IFERROR(...,0)` to avoid errors when `C1` is not found.)", "latency_ms": 14899, "cost": 0.01628025, "task_score": 8, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Correctly identifies the invalid column index causing #VALUE!, provides fix, adds useful logic improvement and IFERROR suggestion. Clear and concise, minor omission of alternative causes." }, { "judge": "google/gemini-3.1-pro-preview", "score": 8, "violates": false, "reasoning": "Identifies the primary structural fault (VLOOKUP index out of bounds) and provides clear, corrected formulas. However, an out-of-bounds VLOOKUP returns #REF!, not #VALUE!, which the response misses." }, { "judge": "x-ai/grok-4.5", "score": 8, "violates": false, "reasoning": "Correctly flags invalid VLOOKUP index as primary fault causing error; offers valid fixes plus optional logic improvement. Minor: actual error is #REF! not #VALUE!." } ] }, { "id": "sp3", "name": "Clean messy data", "prompt": "These postcodes were typed by hand: 'sw1a1aa', 'SW1A 1AA', ' m1 4wb', 'M14WB', 'ec1v9nr'. Give a single Excel formula that normalises any one of them to standard UK format (uppercase, one space before the final three characters). Assume the raw value is in A2.", "output": "Use this single formula in Excel:\n\n```excel\n=LET(pc,UPPER(SUBSTITUTE(TRIM(A2),\" \",\"\")),LEFT(pc,LEN(pc)-3)&\" \"&RIGHT(pc,3))\n```\n\nIt:\n1. Trims leading/trailing spaces,\n2. Removes any internal spaces,\n3. Converts to uppercase,\n4. Inserts exactly one space before the last 3 characters.", "latency_ms": 3888, "cost": 0.004599, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Correct, concise formula meeting requirements; LET requires modern Excel but reasonable assumption; explanation clear and useful." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response provides a flawless, elegant single Excel formula using LET to perfectly normalize the postcodes as requested. The explanation is clear, concise, and highly useful." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Flawless single formula correctly normalizes all given postcodes to standard UK format." } ] }, { "id": "sp4", "name": "Choose the right chart", "prompt": "A shop has 14 months of weekly revenue and wants to see whether sales are seasonal. In under 90 words, say which chart to build, which fields go on which axis, and one thing this chart will NOT tell them.", "output": "Build a **time-series line chart**. Put **week ending date** (or week number in chronological order) on the **x-axis** and **total weekly revenue** on the **y-axis**. \nIf possible, add a second view grouping by **month-of-year** (or week-of-year) to highlight repeating patterns.\n\nWhat it will **not** tell you: whether the pattern is caused by seasonality versus other factors (promotions, stockouts, pricing changes, holidays). It shows pattern, not cause.", "latency_ms": 3590, "cost": 0.00321475, "task_score": 9, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Accurate, clear, meets word limit, gives chart, axes, and limitation; concise and useful for non-technical reader." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, accurately identifies the best chart and axes, provides a highly relevant limitation, and stays well under the 90-word limit." }, { "judge": "x-ai/grok-4.5", "score": 9, "violates": false, "reasoning": "Accurate chart advice for seasonality, meets word limit and all requirements, clear and useful with only minor extra detail." } ] }, { "id": "sp5", "name": "Google Sheets QUERY", "prompt": "In Google Sheets, tab 'Sales' has headers in row 1: Date, Rep, Region, Amount. Write one QUERY() formula returning total Amount per Rep for the North region only, sorted highest first. Return only the formula.", "output": "=QUERY(Sales!A1:D,\"select B, sum(D) where C = 'North' group by B order by sum(D) desc label B 'Rep', sum(D) 'Total Amount'\",1)", "latency_ms": 2092, "cost": 0.0021665, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Correct QUERY formula, proper syntax, sorted descending, grouped by Rep; header row included correctly. Minor: no forbidden constraints violated." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response provides exactly the requested QUERY formula, correctly grouping, filtering, and sorting the data as specified, with no extraneous text." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Correct QUERY formula; filters North, sums by Rep, sorts desc. Returns only the formula as required." } ] }, { "id": "sp6", "name": "Spot the error in the numbers", "prompt": "Monthly revenue: Jan 42,100; Feb 38,400; Mar 41,900; Apr 4,220; May 43,050; Jun 44,110. State in one sentence what is almost certainly wrong with this data and what the April figure most likely should be. Do not speculate about business causes.", "output": "The April value is almost certainly a data-entry error because it is an order of magnitude lower than every other month, and it most likely should be **42,200** (likely missing a zero).", "latency_ms": 2292, "cost": 0.0022715, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 6, "violates": false, "reasoning": "Reasonable identification of anomaly and plausible correction, though guessed exact figure is unverifiable; concise, mostly meets single-sentence requirement despite bolding." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, providing a clear, accurate, one-sentence explanation without speculating on business causes." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "One clear sentence correctly flags April as ~10x too low (data entry) and gives the obvious fix 42,200; no business speculation." } ] }, { "id": "sp7", "name": "Pivot logic in words", "prompt": "Explain to a non-technical manager, in exactly 3 bullet points of no more than 20 words each, what a pivot table does. Do not use the words 'aggregate', 'dimension' or 'axis'.", "output": "- A pivot table quickly summarizes large spreadsheets into clear totals, counts, or averages by chosen categories.\n- You can rearrange rows and columns instantly to view the same data from different business perspectives.\n- It helps spot trends, compare performance, and answer questions without writing complex formulas or creating many separate reports.", "latency_ms": 3354, "cost": 0.0041195, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Meets 3-bullet, ≤20-word constraint, avoids banned words, clear and accurate explanation for a non-technical manager." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly explains pivot tables to a non-technical manager. It strictly follows the exact 3-bullet format, stays under the 20-word limit per bullet, and successfully avoids all forbidden words." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Exact 3 bullets, each <=20 words, no banned terms; accurate, clear, useful summary for non-technical manager." } ] }, { "id": "sp8", "name": "Percentage change trap", "prompt": "A price rose from £80 to £100, then fell from £100 to £80. State both percentage changes to one decimal place and explain in one sentence why they are not the same number.", "output": "- From **£80 to £100**: increase \\(= \\frac{20}{80}\\times 100 = 25.0\\%\\).\n- From **£100 to £80**: decrease \\(= \\frac{20}{100}\\times 100 = 20.0\\%\\).\n\nThey are not the same because each percentage is calculated from a **different starting value** (first from 80, then from 100).", "latency_ms": 2269, "cost": 0.0022505, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Correct percentages to one decimal place, clear explanation in one sentence, concise and well-formatted." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly calculates both percentage changes to one decimal place and provides a clear, accurate one-sentence explanation for why they differ." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Accurate calcs to 1dp; clear one-sentence explanation; fully meets all task needs." } ] }, { "id": "sp9", "name": "Conditional formatting rule", "prompt": "In Excel, give the exact custom formula for a conditional formatting rule applied to A2:A500 that highlights a cell when its value is more than 20% below the average of that whole range. Return the formula and nothing else.", "output": "=A20,\"PAID\",\"UNPAID\")\n```\n\nIf an invoice number in either sheet has a trailing space, this formula can wrongly return **\"UNPAID\"** even when the invoice is actually present in the payments list (because the spaced and unspaced text don’t match exactly).", "latency_ms": 2463, "cost": 0.0023415, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 7, "violates": false, "reasoning": "Correct formula and reasonable explanation, though COUNTIF's exact wildcard matching behavior with trailing spaces is nuanced/inconsistent in Excel, slightly oversimplified but acceptable." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response provides a correct Excel formula and accurately explains the trailing space issue in exactly one sentence, following all instructions perfectly." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Correct COUNTIF formula; accurate one-sentence note on trailing-space mismatch causing false UNPAID. Fully meets all task requirements." } ] }, { "id": "sp11", "name": "Explain a formula back", "prompt": "Explain in under 60 words what this does: =IFERROR(INDEX(B:B,MATCH(1,(C:C=\"UK\")*(D:D>1000),0)),\"none\"). Mention one reason it might be slow on a large sheet.", "output": "It returns the value from column **B** in the first row where **C = \"UK\"** and **D > 1000**. \nIf no such row exists, it returns **\"none\"** (`IFERROR`).\n\nIt may be slow on large sheets because it evaluates full-column ranges (`B:B`, `C:C`, `D:D`), forcing calculations across many rows.", "latency_ms": 2015, "cost": 0.00170975, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Accurate, concise explanation with correct slowness reason, within word limit." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "Perfectly accurate explanation of the formula and its performance issue. It meets the word count constraint (48 words) and is highly clear and useful for a non-technical user." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Accurate explanation of the array formula; under 60 words; clear reason for slowness; fully meets all criteria with no flaws." } ] }, { "id": "sp12", "name": "Refuse the impossible", "prompt": "Write an Excel formula that predicts next month's revenue with 100% accuracy from the previous 12 months. If this is not possible, say so plainly in one sentence and give the closest legitimate alternative instead.", "output": "It’s not possible to predict next month’s revenue with **100% accuracy** from only the prior 12 months, so the closest legitimate alternative is an Excel forecast such as `=FORECAST.ETS(A14,B2:B13,A2:A13,1,1)` (where `A14` is next month’s date, `B2:B13` are the last 12 revenues, and `A2:A13` are their dates).", "latency_ms": 3354, "cost": 0.00310625, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Correctly states impossibility, gives legitimate FORECAST.ETS alternative with clear explanation; concise and accurate, minor lack of detail on seasonality parameters." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response correctly identifies that 100% accuracy is impossible, states this in exactly one sentence as requested, and provides a highly appropriate Excel forecasting formula as the closest alternative." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Correctly states impossibility in one sentence and supplies valid FORECAST.ETS alternative with clear ranges." } ] } ] }