{ "entity": "gemini-3-5-flash", "entity_type": "model", "category": "Spreadsheets", "suite": "suite-2026-08-tasks/spreadsheets", "model": "google/gemini-3.5-flash", "judges": [ "anthropic/claude-sonnet-5", "openai/gpt-5.6-terra", "x-ai/grok-4.5" ], "judge_protocol": "judge-2026-08", "anchors": "solo-2026-08-a", "score": 92, "caps": 0, "cost_usd": 0.2056, "started_at": "2026-08-13T08:58:24.203Z", "finished_at": "2026-08-13T08:59:05.104Z", "tasks": [ { "id": "sp1", "name": "Excel formula from a description", "prompt": "In Excel, column A holds order dates and column B holds order values. Write a single formula for cell D1 that totals only the orders placed in the last full calendar month, relative to today's date. Give the formula and one sentence explaining it. Do not use a pivot table or a helper column.", "output": "`=SUMIFS(B:B, A:A, \">=\"&EOMONTH(TODAY(),-2)+1, A:A, \"<=\"&EOMONTH(TODAY(),-1))`\n\n**Explanation:** \nThis formula uses `SUMIFS` to sum the values in column B where the dates in column A fall between the first day of the previous month (calculated by adding one day to the end of the month two months ago) and the last day of the previous month (calculated using `EOMONTH` with a -1 offset).", "latency_ms": 6521, "cost": 0.0103005, "task_score": 9, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Correct formula, no helper column/pivot, clear one-sentence explanation as required." }, { "judge": "openai/gpt-5.6-terra", "score": 10, "violates": false, "reasoning": "The formula correctly sums column B for dates from the first through last day of the previous calendar month, using no helper column or pivot table. It provides exactly one clear explanatory sentence." }, { "judge": "x-ai/grok-4.5", "score": 9, "violates": false, "reasoning": "Correct SUMIFS formula for prior full month; one clear explanatory sentence; fully meets all task constraints." } ] }, { "id": "sp2", "name": "Fix a broken formula", "prompt": "This Excel formula returns #VALUE! : =SUMIF(A:A,\">\"&TODAY()-30,B:B)+VLOOKUP(C1,Sheet2!A:D,5,FALSE). Identify every fault and give the corrected formula. Sheet2 has columns A to D only.", "output": "There are two faults in your formula that cause errors. Here is the breakdown of the faults and the corrected formula.\n\n### The Faults\n\n1. **Column Index Out of Range (Primary Fault):**\n In your `VLOOKUP` function, you have specified `5` as the column index number: \n `VLOOKUP(C1, Sheet2!A:D, 5, FALSE)`\n However, your lookup range is `Sheet2!A:D`, which only contains **4 columns** (A, B, C, and D). Excel cannot retrieve data from a 5th column that is not included in the range (and does not exist on Sheet2). This causes a `#REF!` error.\n\n2. **Using the `+` Operator (Potential `#VALUE!` Trigger):**\n Using the `+` sign to add the `SUMIF` and `VLOOKUP` results together will trigger a `#VALUE!` error if the `VLOOKUP` returns a text value or if the formula is trying to add a number to an error. \n\n---\n\n### The Corrected Formula\n\nTo fix this, you must change the column index in the `VLOOKUP` to **`4`** (which represents Column D, the last available column in your range). \n\nAdditionally, it is safer to wrap both functions inside a `SUM` function instead of using `+`. The `SUM` function ignores text values, preventing future `#VALUE!` errors if one of the cells contains text.\n\n#### Option 1: Standard Correction (Assuming you want to retrieve Column D)\n```excel\n=SUMIF(A:A,\">\"&TODAY()-30,B:B)+VLOOKUP(C1,Sheet2!A:D,4,FALSE)\n```\n\n#### Option 2: Safer/Robust Version (Prevents `#VALUE!` errors if text is returned)\n```excel\n=SUM(SUMIF(A:A,\">\"&TODAY()-30,B:B), VLOOKUP(C1,Sheet2!A:D,4,FALSE))\n```", "latency_ms": 24779, "cost": 0.0325695, "task_score": 6, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 5, "violates": false, "reasoning": "Correctly identifies column index out of range as root cause, but wrongly claims it causes #REF! not #VALUE!, and SUM doesn't actually ignore errors as claimed—factual inaccuracies undermine correctness despite reasonable structure." }, { "judge": "openai/gpt-5.6-terra", "score": 6, "violates": false, "reasoning": "Correctly identifies the out-of-range VLOOKUP index and provides the appropriate index-4 correction. However, '+' is not inherently faulty, the stated index error produces #REF! rather than #VALUE!, and SUM does not suppress VLOOKUP errors." }, { "judge": "x-ai/grok-4.5", "score": 8, "violates": false, "reasoning": "Correctly spots col_index 5>#REF! on A:D and suggests 4; notes + risk. Minor: user saw #VALUE! not #REF!; extras are fine." } ] }, { "id": "sp3", "name": "Clean messy data", "prompt": "These postcodes were typed by hand: 'sw1a1aa', 'SW1A 1AA', ' m1 4wb', 'M14WB', 'ec1v9nr'. Give a single Excel formula that normalises any one of them to standard UK format (uppercase, one space before the final three characters). Assume the raw value is in A2.", "output": "Here is the Excel formula to normalize the postcode in cell **A2**:\n\n```excel\n=LEFT(UPPER(SUBSTITUTE(A2,\" \",\"\")), LEN(UPPER(SUBSTITUTE(A2,\" \",\"\")))-3) & \" \" & RIGHT(UPPER(SUBSTITUTE(A2,\" \",\"\")), 3)\n```\n\n### How it works:\n1. **`SUBSTITUTE(A2,\" \",\"\")`** removes all existing spaces from the postcode.\n2. **`UPPER(...)`** converts all characters to uppercase.\n3. **`LEFT(..., LEN(...)-3)`** extracts everything except the last 3 characters (the outward code).\n4. **`RIGHT(..., 3)`** extracts exactly the last 3 characters (the inward code).\n5. **`& \" \" &`** joins them back together with a single space in the middle.\n\n---\n\n*Note: If you are using **Excel 365 or Excel 2021+**, you can use this cleaner version using the `LET` function to avoid repeating the cleanup step:*\n\n```excel\n=LET(clean, UPPER(SUBSTITUTE(A2,\" \",\"\")), LEFT(clean, LEN(clean)-3) & \" \" & RIGHT(clean, 3))\n```", "latency_ms": 8340, "cost": 0.0122865, "task_score": 9, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Formula correctly normalizes given examples; also handles leading spaces via SUBSTITUTE removing all spaces. Clear explanation, concise, includes bonus LET version. Minor: doesn't trim non-space whitespace but not required." }, { "judge": "openai/gpt-5.6-terra", "score": 9, "violates": false, "reasoning": "The primary formula correctly uppercases, removes spaces, and inserts one space before the final three characters for every supplied example. It is clear and usable. The optional LET alternative is unnecessary given the request for a single" }, { "judge": "x-ai/grok-4.5", "score": 9, "violates": false, "reasoning": "Correct formula normalizes all examples perfectly. Clear explanation aids busy users. Minor verbosity in extras but fully meets task." } ] }, { "id": "sp4", "name": "Choose the right chart", "prompt": "A shop has 14 months of weekly revenue and wants to see whether sales are seasonal. In under 90 words, say which chart to build, which fields go on which axis, and one thing this chart will NOT tell them.", "output": "Build a **line chart**. \n\n* **X-axis (Horizontal):** Time (Weeks)\n* **Y-axis (Vertical):** Revenue\n\nThis chart will plot weekly revenue over the 14 months, allowing you to visualize recurring peaks and troughs. \n\n**What it will NOT tell you:** \nIt cannot confirm definitive **annual seasonality**. Because you only have 14 months of data, you cannot prove if a trend repeats year-over-year or was just a one-time anomaly. Confirming annual patterns requires at least 24 months of data.", "latency_ms": 8146, "cost": 0.0104505, "task_score": 9, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Correct chart choice, axes, and valid limitation noted; concise, clear, likely under 90 words, useful for non-technical reader." }, { "judge": "openai/gpt-5.6-terra", "score": 10, "violates": false, "reasoning": "It directly recommends an appropriate line chart, assigns weeks to the x-axis and revenue to the y-axis, and clearly states a valid limitation: 14 months cannot establish repeatable annual seasonality. It is concise and under 90 words." }, { "judge": "x-ai/grok-4.5", "score": 9, "violates": false, "reasoning": "Accurate line chart advice with correct axes and valid limitation on 14 months data; under 90 words, clear and useful." } ] }, { "id": "sp5", "name": "Google Sheets QUERY", "prompt": "In Google Sheets, tab 'Sales' has headers in row 1: Date, Rep, Region, Amount. Write one QUERY() formula returning total Amount per Rep for the North region only, sorted highest first. Return only the formula.", "output": "`=QUERY(Sales!A:D, \"SELECT B, SUM(D) WHERE C = 'North' GROUP BY B ORDER BY SUM(D) DESC\", 1)`", "latency_ms": 3592, "cost": 0.00495, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Correct formula logic, appropriate GROUP BY/ORDER BY, header handling. Minor concern: header label true for row1 count, but fine since data starts row1." }, { "judge": "openai/gpt-5.6-terra", "score": 10, "violates": false, "reasoning": "Correct QUERY syntax: filters Region to North, groups by Rep, sums Amount, sorts descending, and specifies one header row. The response contains only the requested formula, with harmless code formatting." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Correct QUERY formula matching all specs; only the formula returned as required." } ] }, { "id": "sp6", "name": "Spot the error in the numbers", "prompt": "Monthly revenue: Jan 42,100; Feb 38,400; Mar 41,900; Apr 4,220; May 43,050; Jun 44,110. State in one sentence what is almost certainly wrong with this data and what the April figure most likely should be. Do not speculate about business causes.", "output": "The April revenue figure is almost certainly missing a trailing zero due to a data entry error, and it should most likely be 42,200.", "latency_ms": 4471, "cost": 0.0054915, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Correctly identifies likely data entry error and gives plausible corrected figure in one sentence, concise and follows instructions without speculating on causes." }, { "judge": "openai/gpt-5.6-terra", "score": 10, "violates": false, "reasoning": "Correctly identifies April as an apparent order-of-magnitude data-entry anomaly and infers 42,200, consistent with neighboring monthly revenues. It is one sentence and does not speculate about business causes." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "One clear sentence correctly flags April as missing a zero (typo) and gives the obvious fix 42,200; no business speculation." } ] }, { "id": "sp7", "name": "Pivot logic in words", "prompt": "Explain to a non-technical manager, in exactly 3 bullet points of no more than 20 words each, what a pivot table does. Do not use the words 'aggregate', 'dimension' or 'axis'.", "output": "* It takes large, messy sheets of data and instantly summarizes them into clean, readable reports.\n* You can quickly reorganize and view your information from different angles using simple drag-and-drop tools.\n* It helps you spot trends, compare numbers, and answer key business questions without writing complex formulas.", "latency_ms": 6456, "cost": 0.0091575, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Three bullets, each under 20 words, avoids banned words, clear and useful explanation for a manager." }, { "judge": "openai/gpt-5.6-terra", "score": 10, "violates": false, "reasoning": "Exactly three bullets, each under 20 words, with no banned terms. Clearly and accurately explains summarizing, reorganizing, comparing, and analyzing data for a non-technical manager." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Exact 3 bullets, each ≤20 words, no banned terms; accurate, clear, useful summary." } ] }, { "id": "sp8", "name": "Percentage change trap", "prompt": "A price rose from £80 to £100, then fell from £100 to £80. State both percentage changes to one decimal place and explain in one sentence why they are not the same number.", "output": "* **Percentage increase (from £80 to £100):** 25.0%\n* **Percentage decrease (from £100 to £80):** 20.0%\n\n**Explanation:** \nThe percentage changes are different because the same absolute change of £20 is calculated relative to two different starting values (£80 for the increase and £100 for the decrease).", "latency_ms": 3724, "cost": 0.00591, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Correct percentages, one decimal place, clear one-sentence explanation of base difference; concise and accurate." }, { "judge": "openai/gpt-5.6-terra", "score": 10, "violates": false, "reasoning": "Both percentages are correct to one decimal place, and the explanation accurately states that each £20 change uses a different starting value as its denominator. Clear, concise, and fully follows the requested format." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Exact correct percentages to 1dp; single clear sentence on different bases; fully meets all task requirements." } ] }, { "id": "sp9", "name": "Conditional formatting rule", "prompt": "In Excel, give the exact custom formula for a conditional formatting rule applied to A2:A500 that highlights a cell when its value is more than 20% below the average of that whole range. Return the formula and nothing else.", "output": "=A21000),0)),\"none\"). Mention one reason it might be slow on a large sheet.", "output": "This formula finds the first value in column B where column C is \"UK\" and column D is greater than 1000, returning \"none\" if unmatched. \n\nIt can be slow because referencing entire columns (like `C:C`) forces Excel to evaluate over a million rows for the array calculation.", "latency_ms": 5325, "cost": 0.008115, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Accurate explanation, mentions array formula entered normally may not work but not noted; concise, under 60 words, gives slowness reason correctly." }, { "judge": "openai/gpt-5.6-terra", "score": 10, "violates": false, "reasoning": "Accurately explains that it returns the first column B value meeting both conditions, otherwise \"none,\" and correctly identifies full-column array evaluation as a large-sheet performance concern. It is clear, concise, and under 60 words." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Accurate explanation of the array formula; under 60 words; correctly notes full-column refs cause slowness. Clear and complete." } ] }, { "id": "sp12", "name": "Refuse the impossible", "prompt": "Write an Excel formula that predicts next month's revenue with 100% accuracy from the previous 12 months. If this is not possible, say so plainly in one sentence and give the closest legitimate alternative instead.", "output": "It is impossible to predict next month's revenue with 100% accuracy because future market conditions, customer behavior, and external economic factors cannot be known with absolute certainty. \n\nThe closest legitimate alternative is to use Excel's **`FORECAST.LINEAR`** formula, which uses linear regression to project next month's revenue based on the trend of your previous 12 months.\n\n### The Formula\nAssuming your past 12 months of dates are in cells **`A2:A13`**, your past 12 months of revenue are in cells **`B2:B13`**, and the date for next month is in cell **`A14`**:\n\n```excel\n=FORECAST.LINEAR(A14, B2:B13, A2:A13)\n```\n\n### How it works:\n* **`A14`** (x): The target date you want to predict (next month).\n* **`B2:B13`** (known_y's): The historical revenue data.\n* **`A2:A13`** (known_x's): The historical timeline/dates. \n\n*(Note: If you have multiple years of data and want to account for seasonal patterns like holiday surges, you should use `=FORECAST.ETS` instead, but this requires at least 24 months of data to work accurately).*", "latency_ms": 9451, "cost": 0.012534, "task_score": 9, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Correctly states impossibility, provides accurate FORECAST.LINEAR formula and clear explanation, mentions FORECAST.ETS caveat. Meets task requirements well, clear and useful." }, { "judge": "openai/gpt-5.6-terra", "score": 8, "violates": false, "reasoning": "Correctly states 100% accuracy is impossible and provides a valid FORECAST.LINEAR formula with clear assumptions. The only notable issue is the oversimplified claim that FORECAST.ETS requires at least 24 months; it does not strictly require" }, { "judge": "x-ai/grok-4.5", "score": 9, "violates": false, "reasoning": "Correctly states impossibility in one plain sentence then gives accurate FORECAST.LINEAR alternative with clear setup; highly useful and polished." } ] } ] }