{
 "entity": "gpt-5-5",
 "entity_type": "model",
 "category": "Spreadsheets",
 "suite": "suite-2026-08-tasks/spreadsheets",
 "model": "openai/gpt-5.5",
 "judges": [
  "anthropic/claude-sonnet-5",
  "google/gemini-3.1-pro-preview",
  "x-ai/grok-4.5"
 ],
 "judge_protocol": "judge-2026-08",
 "anchors": "solo-2026-08-a",
 "score": 97,
 "caps": 0,
 "cost_usd": 0.228,
 "started_at": "2026-08-13T08:55:15.274Z",
 "finished_at": "2026-08-13T08:55:47.555Z",
 "tasks": [
  {
   "id": "sp1",
   "name": "Excel formula from a description",
   "prompt": "In Excel, column A holds order dates and column B holds order values. Write a single formula for cell D1 that totals only the orders placed in the last full calendar month, relative to today's date. Give the formula and one sentence explaining it. Do not use a pivot table or a helper column.",
   "output": "```excel\n=SUMIFS(B:B,A:A,\">=\"&EOMONTH(TODAY(),-2)+1,A:A,\"<\"&EOMONTH(TODAY(),-1)+1)\n```\n\nThis sums values in column B where the corresponding order date in column A is on or after the first day of the previous month and before the first day of the current month.",
   "latency_ms": 4661,
   "cost": 0.006455,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Formula correctly sums last full calendar month using SUMIFS with EOMONTH boundaries, no pivot/helper column, concise explanation."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response provides a flawless, accurate Excel formula that perfectly meets all constraints, accompanied by exactly one clear explanatory sentence."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Correct SUMIFS with EOMONTH for prior full month; exact format and clear one-sentence explanation."
    }
   ]
  },
  {
   "id": "sp2",
   "name": "Fix a broken formula",
   "prompt": "This Excel formula returns #VALUE! : =SUMIF(A:A,\">\"&TODAY()-30,B:B)+VLOOKUP(C1,Sheet2!A:D,5,FALSE). Identify every fault and give the corrected formula. Sheet2 has columns A to D only.",
   "output": "Faults:\n\n1. **`VLOOKUP` column index is invalid**  \n   `Sheet2!A:D` has only **4 columns**, so `VLOOKUP(...,5,...)` is impossible. The largest valid column index is `4`.\n\n2. **The date criterion should be parenthesized for clarity**  \n   Use `\">\" & (TODAY()-30)`.\n\nCorrected formula:\n\n```excel\n=SUMIF(A:A,\">\"&(TODAY()-30),B:B)+VLOOKUP(C1,Sheet2!A:D,4,FALSE)\n```\n\nIf you truly need the 5th column, then the lookup range would have to include column E, e.g. `Sheet2!A:E`, but you said Sheet2 only has columns A to D, so `4` is the correct maximum column index.",
   "latency_ms": 17763,
   "cost": 0.036855,
   "task_score": 7,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 7,
     "violates": false,
     "reasoning": "Correctly identifies main fault (invalid column index 5) and provides valid corrected formula. Minor point about parentheses is unnecessary but harmless. Clear, concise, useful."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 5,
     "violates": false,
     "reasoning": "Identifies the #REF! error (column index 5 in A:D) but fails to explain the #VALUE! error, which occurs because VLOOKUP likely returns text that cannot be added (+) to the SUMIF number."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 7,
     "violates": false,
     "reasoning": "Correctly fixes both issues in formula; understates date-precedence as mere clarity not a #VALUE! cause, so incomplete fault list."
    }
   ]
  },
  {
   "id": "sp3",
   "name": "Clean messy data",
   "prompt": "These postcodes were typed by hand: 'sw1a1aa', 'SW1A 1AA', ' m1 4wb', 'M14WB', 'ec1v9nr'. Give a single Excel formula that normalises any one of them to standard UK format (uppercase, one space before the final three characters). Assume the raw value is in A2.",
   "output": "```excel\n=LET(p,UPPER(SUBSTITUTE(TRIM(A2),\" \",\"\")),LEFT(p,LEN(p)-3)&\" \"&RIGHT(p,3))\n```",
   "latency_ms": 6165,
   "cost": 0.0102,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Correct, concise formula using LET, handles case/spacing normalization properly; minor lack of explanation but meets task requirements well."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The formula correctly removes all spaces, converts to uppercase, and inserts a single space before the last three characters using the LET function. It perfectly meets all requirements."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Formula correctly normalizes all examples to standard UK postcode format via trim, despace, upper, and insert space."
    }
   ]
  },
  {
   "id": "sp4",
   "name": "Choose the right chart",
   "prompt": "A shop has 14 months of weekly revenue and wants to see whether sales are seasonal. In under 90 words, say which chart to build, which fields go on which axis, and one thing this chart will NOT tell them.",
   "output": "Build a **time-series line chart**.\n\n- **X-axis:** Week (week start date or week number in chronological order)  \n- **Y-axis:** Weekly revenue  \n- **Line:** Revenue over the 14-month period\n\nThis will show repeating peaks, dips, or trends that may suggest seasonality. It will **not** tell them why sales rise or fall, or prove that the pattern is truly seasonal rather than caused by promotions, holidays, stock issues, or random variation.",
   "latency_ms": 4730,
   "cost": 0.006835,
   "task_score": 9,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Accurate, clear, meets word limit, correctly identifies chart, axes, and a valid limitation; concise and useful."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "Meets all constraints, including the word limit. The chart recommendation, axis mappings, and limitation are perfectly accurate, clear, and highly useful for a non-technical user."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 9,
     "violates": false,
     "reasoning": "Accurate chart advice, meets all task points under 90 words, clear and useful for non-technical user."
    }
   ]
  },
  {
   "id": "sp5",
   "name": "Google Sheets QUERY",
   "prompt": "In Google Sheets, tab 'Sales' has headers in row 1: Date, Rep, Region, Amount. Write one QUERY() formula returning total Amount per Rep for the North region only, sorted highest first. Return only the formula.",
   "output": "=QUERY(Sales!A:D,\"select B, sum(D) where C = 'North' group by B order by sum(D) desc label sum(D) 'Total Amount'\",1)",
   "latency_ms": 2790,
   "cost": 0.00534,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Correct QUERY formula, proper syntax, sorted descending, labeled correctly. Minor concern: header row handling with A:D range but generally valid and functional."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response provides exactly the requested QUERY formula, perfectly addressing all requirements (filtering, grouping, sorting) and returning only the formula as instructed."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Correct QUERY syntax and logic for the exact request; returns solely the formula as required; clear and immediately usable."
    }
   ]
  },
  {
   "id": "sp6",
   "name": "Spot the error in the numbers",
   "prompt": "Monthly revenue: Jan 42,100; Feb 38,400; Mar 41,900; Apr 4,220; May 43,050; Jun 44,110. State in one sentence what is almost certainly wrong with this data and what the April figure most likely should be. Do not speculate about business causes.",
   "output": "The April revenue is almost certainly a data-entry error—4,220 is off by a factor of ten—and it most likely should be 42,200.",
   "latency_ms": 5874,
   "cost": 0.00457,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Correctly identifies decimal/factor-of-ten error, gives plausible corrected figure, one sentence, avoids business speculation. Minor: could justify estimate better."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response perfectly follows all instructions, providing a clear, accurate, one-sentence explanation of the likely data entry error without speculating on business causes."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Perfect one-sentence answer correctly identifying the factor-of-ten error and likely true value; fully meets all criteria."
    }
   ]
  },
  {
   "id": "sp7",
   "name": "Pivot logic in words",
   "prompt": "Explain to a non-technical manager, in exactly 3 bullet points of no more than 20 words each, what a pivot table does. Do not use the words 'aggregate', 'dimension' or 'axis'.",
   "output": "- Summarizes large lists into clear totals, counts, or averages.\n- Lets you quickly compare results by categories like team, month, or product.\n- Helps spot patterns without changing the original data.",
   "latency_ms": 4919,
   "cost": 0.0028,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Clear, correct, meets 3 bullets, under 20 words each, avoids banned words. Very concise and useful for non-technical manager."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "Follows all constraints perfectly. The explanation is highly accessible, accurate, and concise, making it ideal for a non-technical manager."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Exact 3 bullets, each under 20 words, no banned terms; accurate and clear for non-technical audience."
    }
   ]
  },
  {
   "id": "sp8",
   "name": "Percentage change trap",
   "prompt": "A price rose from £80 to £100, then fell from £100 to £80. State both percentage changes to one decimal place and explain in one sentence why they are not the same number.",
   "output": "The price increased by **25.0%** from £80 to £100, then decreased by **20.0%** from £100 to £80**.**\n\nThey are not the same percentage because each change is calculated relative to a different starting value: £80 for the rise and £100 for the fall.",
   "latency_ms": 3041,
   "cost": 0.00332,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Correct percentages (25.0%, 20.0%), accurate one-sentence explanation, clear and concise formatting."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response perfectly calculates both percentage changes to one decimal place and accurately explains the difference in exactly one sentence."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Correct percentages to 1dp, accurate one-sentence explanation of base difference; fully follows task, clear and concise."
    }
   ]
  },
  {
   "id": "sp9",
   "name": "Conditional formatting rule",
   "prompt": "In Excel, give the exact custom formula for a conditional formatting rule applied to A2:A500 that highlights a cell when its value is more than 20% below the average of that whole range. Return the formula and nothing else.",
   "output": "=A2<0.8*AVERAGE($A$2:$A$500)",
   "latency_ms": 2640,
   "cost": 0.005125,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Correct formula for conditional formatting, relative row reference appropriate for range, matches instruction to return only formula."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response provides the exact, correct Excel formula and strictly follows the negative constraint to return nothing else."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Exact correct CF formula; only the formula returned as required."
    }
   ]
  },
  {
   "id": "sp10",
   "name": "Reconcile two lists",
   "prompt": "You have invoice numbers in Sheet1!A:A and payments in Sheet2!A:A. Give one Excel formula for Sheet1!B2 that returns 'PAID' or 'UNPAID', and state in one sentence what it will wrongly report if an invoice number has a trailing space.",
   "output": "In `Sheet1!B2` use:\n\n```excel\n=IF(ISNUMBER(MATCH(A2,Sheet2!A:A,0)),\"PAID\",\"UNPAID\")\n```\n\nIf the invoice number has a trailing space on one sheet but not the other, this exact-match formula may wrongly report `UNPAID` even though the invoice is actually paid.",
   "latency_ms": 11420,
   "cost": 0.01827,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Correct, valid formula and accurate explanation of trailing space issue; concise and clear, meets task requirements well."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response provides a correct and efficient Excel formula, and accurately explains the trailing space issue in exactly one sentence as requested."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Flawless exact-match formula and precise one-sentence note on trailing-space false UNPAID."
    }
   ]
  },
  {
   "id": "sp11",
   "name": "Explain a formula back",
   "prompt": "Explain in under 60 words what this does: =IFERROR(INDEX(B:B,MATCH(1,(C:C=\"UK\")*(D:D>1000),0)),\"none\"). Mention one reason it might be slow on a large sheet.",
   "output": "Returns the value from column B on the first row where column C equals “UK” and column D is greater than 1000. If no match is found, it returns “none”. It may be slow because it performs array calculations over entire columns, scanning over a million rows.",
   "latency_ms": 3693,
   "cost": 0.004245,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Accurate explanation, mentions slowness reason (full-column array calc), concise. Word count near 60 but likely under limit; clear and useful."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response accurately explains the formula, correctly identifies the performance issue (array calculations over entire columns), and strictly adheres to the under 60-word limit constraint."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Accurate explanation of the array INDEX/MATCH/IFERROR formula; under 60 words; correctly notes full-column scan as slowdown cause; clear and complete."
    }
   ]
  },
  {
   "id": "sp12",
   "name": "Refuse the impossible",
   "prompt": "Write an Excel formula that predicts next month's revenue with 100% accuracy from the previous 12 months. If this is not possible, say so plainly in one sentence and give the closest legitimate alternative instead.",
   "output": "It isn’t possible to predict next month’s revenue with 100% accuracy from only the previous 12 months; the closest legitimate Excel alternative is a forecast such as `=FORECAST.ETS(EDATE(MAX(A2:A13),1),B2:B13,A2:A13)`, assuming dates are in `A2:A13` and revenues are in `B2:B13`.",
   "latency_ms": 5179,
   "cost": 0.007225,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Correctly states impossibility plainly, provides legitimate FORECAST.ETS formula with correct range assumptions. Concise, accurate, useful."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response perfectly follows all instructions, providing a single sentence that plainly states the impossibility of 100% accuracy and offers a highly accurate and useful Excel forecasting formula as the alternative."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Correctly states impossibility in one sentence and supplies valid FORECAST.ETS alternative with clear assumptions."
    }
   ]
  }
 ]
}