{
 "entity": "gemini-3-1-pro",
 "entity_type": "model",
 "category": "Spreadsheets",
 "suite": "suite-2026-08-tasks/spreadsheets",
 "model": "google/gemini-3.1-pro-preview",
 "judges": [
  "anthropic/claude-sonnet-5",
  "openai/gpt-5.6-terra",
  "x-ai/grok-4.5"
 ],
 "judge_protocol": "judge-2026-08",
 "anchors": "solo-2026-08-a",
 "score": 93,
 "caps": 3,
 "cost_usd": 0.2547,
 "started_at": "2026-08-13T08:57:36.186Z",
 "finished_at": "2026-08-13T08:58:24.200Z",
 "tasks": [
  {
   "id": "sp1",
   "name": "Excel formula from a description",
   "prompt": "In Excel, column A holds order dates and column B holds order values. Write a single formula for cell D1 that totals only the orders placed in the last full calendar month, relative to today's date. Give the formula and one sentence explaining it. Do not use a pivot table or a helper column.",
   "output": "**Formula:**\n`=SUMIFS(B:B, A:A, \">=\"&EOMONTH(TODAY(),-2)+1, A:A, \"<=\"&EOMONTH(TODAY(),-1))`\n\n**Explanation:**\nThis formula uses the `SUMIFS` function to add the values in column B where the corresponding dates in column A fall between the first and last days of the previous month, which are calculated dynamically using the `EOMONTH` function relative to `TODAY()`.",
   "latency_ms": 9916,
   "cost": 0.01455,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Correct formula, no helper column/pivot, clear explanation, meets constraints and conciseness well."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "The formula correctly sums values whose dates fall from the first through last day of the previous calendar month, using dynamic EOMONTH boundaries. It is a single formula with no helper column or pivot table, and the explanation is clear."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Correct SUMIFS with EOMONTH for prior full month; single formula plus one clear sentence; fully meets all requirements."
    }
   ]
  },
  {
   "id": "sp2",
   "name": "Fix a broken formula",
   "prompt": "This Excel formula returns #VALUE! : =SUMIF(A:A,\">\"&TODAY()-30,B:B)+VLOOKUP(C1,Sheet2!A:D,5,FALSE). Identify every fault and give the corrected formula. Sheet2 has columns A to D only.",
   "output": "Here is the breakdown of every fault in your formula, followed by the corrected versions depending on what you are trying to achieve.\n\n### **The Faults**\n\n**1. The Column Index is out of bounds (Causes a `#REF!` error)**\nIn your `VLOOKUP(C1,Sheet2!A:D,5,FALSE)`, you specified the table array as `Sheet2!A:D`. This range contains exactly **4 columns** (A=1, B=2, C=3, D=4). However, you asked Excel to return the value from column **5**. Because column 5 does not exist in the defined range, this part of the formula will return a `#REF!` error. \n\n**2. Mathematical addition of Text (Causes the `#VALUE!` error)**\nYou are using the plus sign (`+`) to add the result of the `SUMIF` to the result of the `VLOOKUP`. The `+` operator strictly requires **numbers**. If your `VLOOKUP` returns a text string, a blank space, or an empty string (`\"\"`), Excel cannot do math with it, which results in the `#VALUE!` error. \n\n**3. No Error Handling for missing data (Causes an `#N/A` error)**\nIf the value in cell `C1` does not exist in `Sheet2!A:A`, the `VLOOKUP` will return `#N/A`. If you try to add `#N/A` to your `SUMIF` total, the entire formula will break and display `#N/A`.\n\n---\n\n### **The Corrected Formulas**\n\nTo fix this, you must change the column index to **4** (or 1, 2, or 3) and handle the data types correctly. Choose the formula below that matches your goal:\n\n#### **Option 1: If you are trying to ADD two numbers together**\nIf column D in Sheet2 contains numbers and you want to mathematically add them to your SUMIF total, use this formula. It changes the column index to 4 and wraps the VLOOKUP in an `IFERROR` so that if the lookup fails, it adds a zero instead of breaking the formula.\n\n`=SUMIF(A:A,\">\"&TODAY()-30,B:B) + IFERROR(VLOOKUP(C1,Sheet2!A:D,4,FALSE), 0)`\n\n*(Note: If the VLOOKUP successfully finds a value, but that value is text, you will still get a #VALUE! error. Ensure Column D contains actual numbers).*\n\n#### **Option 2: If you are trying to COMBINE text and numbers (Concatenation)**\nIf you want the cell to display the numeric total followed by a text label pulled from Sheet2 (for example: `500 USD` or `500 Completed`), you must replace the plus sign (`+`) with an ampersand (`&`). \n\n`=SUMIF(A:A,\">\"&TODAY()-30,B:B) & \" \" & IFERROR(VLOOKUP(C1,Sheet2!A:D,4,FALSE), \"\")`\n\n*(Note: The `\" \" &` adds a space between your SUMIF number and your VLOOKUP text so they don't mash together).*",
   "latency_ms": 22296,
   "cost": 0.032518,
   "task_score": 6,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 6,
     "violates": false,
     "reasoning": "Correctly identifies column index and error-handling issues, but wrongly claims out-of-range index causes #REF! (actually #VALUE!... but in +VLOOKUP context it's #REF! is plausible); slightly verbose but generally accurate and useful."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 6,
     "violates": false,
     "reasoning": "Correctly identifies invalid VLOOKUP column 5 and supplies workable formulas. However, with Sheet2!A:D, the stated formula produces #REF!, not #VALUE!, which it should explicitly flag. Text-addition and missing-lookup issues are conditional"
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 8,
     "violates": false,
     "reasoning": "Accurately flags col-index #REF! and + on text as #VALUE! source; supplies solid fixes. Minor: original error is #VALUE! yet primary fault yields #REF!."
    }
   ]
  },
  {
   "id": "sp3",
   "name": "Clean messy data",
   "prompt": "These postcodes were typed by hand: 'sw1a1aa', 'SW1A 1AA', ' m1 4wb', 'M14WB', 'ec1v9nr'. Give a single Excel formula that normalises any one of them to standard UK format (uppercase, one space before the final three characters). Assume the raw value is in A2.",
   "output": "Here is the single formula that will work in all versions of Excel:\n\n`=LEFT(UPPER(SUBSTITUTE(A2, \" \", \"\")), LEN(SUBSTITUTE(A2, \" \", \"\")) - 3) & \" \" & RIGHT(UPPER(SUBSTITUTE(A2, \" \", \"\")), 3)`\n\n### How it works:\n1. **`SUBSTITUTE(A2, \" \", \"\")`**: Strips out any existing spaces (whether they are leading, trailing, or in the middle).\n2. **`UPPER(...)`**: Converts the entire string to uppercase.\n3. **`RIGHT(..., 3)`**: UK postcodes *always* end with exactly 3 characters (the \"incode\"). This extracts those final 3 characters.\n4. **`LEFT(..., LEN(...) - 3)`**: Extracts everything *except* those final 3 characters (the \"outcode\").\n5. **`& \" \" &`**: Joins the outcode and the incode back together with exactly one space in the middle.\n\n*(Note: If you are using a newer version of Excel like Office 365 or Excel 2021, you can use the `LET` function to make the formula shorter and more efficient: `=LET(pc, UPPER(SUBSTITUTE(A2, \" \", \"\")), LEFT(pc, LEN(pc)-3) & \" \" & RIGHT(pc, 3))`)*",
   "latency_ms": 16011,
   "cost": 0.024578,
   "task_score": 8,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Correct, well-explained formula handling spaces/case; leading space trimmed via SUBSTITUTE. Slightly verbose but clear and useful, meets task requirements."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 5,
     "violates": true,
     "reasoning": "The primary formula correctly uppercases, removes spaces, and inserts one space before the final three characters for every supplied example. However, the response includes a second optional LET formula despite requesting a single formula."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Formula correctly normalizes all examples to UK format; clear single formula with accurate explanation; highly useful and concise."
    }
   ]
  },
  {
   "id": "sp4",
   "name": "Choose the right chart",
   "prompt": "A shop has 14 months of weekly revenue and wants to see whether sales are seasonal. In under 90 words, say which chart to build, which fields go on which axis, and one thing this chart will NOT tell them.",
   "output": "Build a **line chart**. \n\n* **X-axis:** Time (Weeks)\n* **Y-axis:** Revenue ($)\n\n**What it will NOT tell them:** \nIt will not definitively prove seasonality. Because 14 months only provides two overlapping months for a year-over-year comparison, the chart cannot confirm if a specific revenue spike is a recurring seasonal pattern or just a one-off anomaly (like a unique promotion, a viral event, or unusual weather). You generally need at least two full years of data to confidently confirm true seasonality.",
   "latency_ms": 8208,
   "cost": 0.010382,
   "task_score": 9,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 5,
     "violates": true,
     "reasoning": "Good content but exceeds 90-word limit (~90-100 words), violating explicit constraint. Otherwise accurate and clear advice."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Correctly recommends a line chart with weeks on the x-axis and revenue on the y-axis. Clearly states a key limitation: 14 months cannot establish recurring seasonality. It is accurate, useful, concise, and under 90 words."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 9,
     "violates": false,
     "reasoning": "Accurate chart advice, meets word limit and all requirements, clear and useful with minor extra detail."
    }
   ]
  },
  {
   "id": "sp5",
   "name": "Google Sheets QUERY",
   "prompt": "In Google Sheets, tab 'Sales' has headers in row 1: Date, Rep, Region, Amount. Write one QUERY() formula returning total Amount per Rep for the North region only, sorted highest first. Return only the formula.",
   "output": "=QUERY(Sales!A:D, \"SELECT B, SUM(D) WHERE C = 'North' GROUP BY B ORDER BY SUM(D) DESC\", 1)",
   "latency_ms": 4547,
   "cost": 0.005784,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Correct, functional QUERY formula meeting requirements; minor nit: label header omitted but functionally fine."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Correct QUERY syntax: filters Region column C to North, groups by Rep column B, sums Amount column D, sorts descending by total, and uses one header row. Returns only the formula."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Correct QUERY formula matching all requirements exactly; only the formula returned."
    }
   ]
  },
  {
   "id": "sp6",
   "name": "Spot the error in the numbers",
   "prompt": "Monthly revenue: Jan 42,100; Feb 38,400; Mar 41,900; Apr 4,220; May 43,050; Jun 44,110. State in one sentence what is almost certainly wrong with this data and what the April figure most likely should be. Do not speculate about business causes.",
   "output": "The April revenue figure is almost certainly missing a zero due to a data entry error, meaning it most likely should be 42,200 to align with the rest of the dataset.",
   "latency_ms": 6132,
   "cost": 0.008018,
   "task_score": 9,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 6,
     "violates": false,
     "reasoning": "Correctly identifies missing digit error; estimate reasonable though arbitrary; concise, meets one-sentence format, avoids business speculation."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "It correctly identifies April as an apparent missing-zero outlier and gives the most likely corrected figure, 42,200, in one clear sentence without speculating about business causes."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 9,
     "violates": false,
     "reasoning": "Correctly identifies missing zero as data error and proposes 42200; one clear sentence; fully follows constraints without business speculation."
    }
   ]
  },
  {
   "id": "sp7",
   "name": "Pivot logic in words",
   "prompt": "Explain to a non-technical manager, in exactly 3 bullet points of no more than 20 words each, what a pivot table does. Do not use the words 'aggregate', 'dimension' or 'axis'.",
   "output": "* It turns massive spreadsheets of raw data into small, clear summary tables.\n* It automatically groups related items together to calculate totals, averages, or counts instantly.\n* It lets you easily rearrange rows and columns to answer different business questions quickly.",
   "latency_ms": 12381,
   "cost": 0.020106,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Clear, accurate, concise bullets under 20 words each, avoids banned terms, non-technical friendly explanation of pivot tables."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Exactly three bullets, each under 20 words, with no banned terms. Accurate, manager-friendly explanation covering summarization, calculations, and flexible rearrangement."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Exact 3 bullets under 20 words each; no banned terms; accurate, clear, useful summary for managers."
    }
   ]
  },
  {
   "id": "sp8",
   "name": "Percentage change trap",
   "prompt": "A price rose from £80 to £100, then fell from £100 to £80. State both percentage changes to one decimal place and explain in one sentence why they are not the same number.",
   "output": "**Percentage Changes:**\n*   **Increase:** 25.0% (from £80 to £100)\n*   **Decrease:** 20.0% (from £100 to £80)\n\n**Explanation:**\nThe percentage changes are different because the identical £20 absolute change is calculated relative to a different starting price in each case (£80 for the increase versus £100 for the decrease).",
   "latency_ms": 6575,
   "cost": 0.009464,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Correct percentages, one decimal place, clear one-sentence explanation. Minor formatting excess but accurate and useful."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Both changes are calculated correctly to one decimal place, and the explanation accurately identifies the different starting values (bases). The response is clear, concise, and fully follows the requested format and one-sentence explanation"
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Correct 25.0% and 20.0% calculations; one-sentence explanation accurate; fully follows format and decimal requirements; clear and useful."
    }
   ]
  },
  {
   "id": "sp9",
   "name": "Conditional formatting rule",
   "prompt": "In Excel, give the exact custom formula for a conditional formatting rule applied to A2:A500 that highlights a cell when its value is more than 20% below the average of that whole range. Return the formula and nothing else.",
   "output": "=A2<0.8*AVERAGE($A$2:$A$500)",
   "latency_ms": 5855,
   "cost": 0.007446,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Correct formula for conditional formatting with proper relative/absolute references, returned alone as instructed."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "The formula correctly compares each relative cell in A2:A500 to 80% of the absolute average of the entire range, and returns only the formula as required."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Exact correct formula; only the formula returned as required."
    }
   ]
  },
  {
   "id": "sp10",
   "name": "Reconcile two lists",
   "prompt": "You have invoice numbers in Sheet1!A:A and payments in Sheet2!A:A. Give one Excel formula for Sheet1!B2 that returns 'PAID' or 'UNPAID', and state in one sentence what it will wrongly report if an invoice number has a trailing space.",
   "output": "**Formula:**\n`=IF(COUNTIF(Sheet2!A:A, A2)>0, \"PAID\", \"UNPAID\")`\n\n**Explanation:**\nIf an invoice number has a trailing space in one sheet but not the other, the formula will fail to recognize them as a match and will wrongly report **'UNPAID'**.",
   "latency_ms": 7760,
   "cost": 0.01084,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Correct formula and accurate explanation; concise and meets task requirements, though COUNTIF wildcard behavior nuance not fully addressed."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "The formula correctly checks whether the Sheet1 invoice number appears in Sheet2 and returns the required labels. The one-sentence explanation accurately identifies a trailing-space mismatch as causing a false UNPAID result."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Correct COUNTIF formula; accurately states trailing-space mismatch wrongly yields UNPAID; fully meets all task requirements clearly."
    }
   ]
  },
  {
   "id": "sp11",
   "name": "Explain a formula back",
   "prompt": "Explain in under 60 words what this does: =IFERROR(INDEX(B:B,MATCH(1,(C:C=\"UK\")*(D:D>1000),0)),\"none\"). Mention one reason it might be slow on a large sheet.",
   "output": "This formula returns the first value in column B where column C is \"UK\" and column D is over 1000. If no match exists, it outputs \"none\". \n\n**Why it's slow:** Evaluating array conditions across entire columns (`C:C` and `D:D`) forces Excel to process over a million empty rows unnecessarily.",
   "latency_ms": 7797,
   "cost": 0.011876,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Accurate explanation, notes array formula/full-column reference slowdown, concise, under 60 words. Note: it's a CSE array formula not mentioned explicitly, minor omission."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Accurately explains the first matching value, the fallback result, and a valid performance issue from whole-column array evaluation. It is clear, useful, and well under 60 words."
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 10,
     "violates": false,
     "reasoning": "Accurate explanation of the array formula; under 60 words; clear slow-reason given; fully meets all criteria with no flaws."
    }
   ]
  },
  {
   "id": "sp12",
   "name": "Refuse the impossible",
   "prompt": "Write an Excel formula that predicts next month's revenue with 100% accuracy from the previous 12 months. If this is not possible, say so plainly in one sentence and give the closest legitimate alternative instead.",
   "output": "It is impossible to predict future revenue with 100% accuracy because no mathematical formula can account for unpredictable real-world variables and market changes. \n\nThe closest legitimate alternative is to use Excel's built-in forecasting functions to project a statistically probable estimate based on your historical trends. \n\nAssuming your previous 12 months (dates or period numbers 1-12) are in cells **A2:A13**, your historical revenue is in **B2:B13**, and the future date/period you want to predict is in **A14**, here are the two best options:\n\n**1. The Standard Trend (Linear Regression)**\nIf you want to project the general upward or downward trend of your revenue, use this formula:\n`=FORECAST.LINEAR(A14, B2:B13, A2:A13)`\n\n**2. The Seasonal Trend (Exponential Smoothing)**\nIf your revenue has seasonal fluctuations (e.g., sales always spike in certain months), Excel has a more advanced formula that detects and accounts for these patterns:\n`=FORECAST.ETS(A14, B2:B13, A2:A13, 1)`",
   "latency_ms": 11753,
   "cost": 0.016412,
   "task_score": 9,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Correctly states impossibility, offers legitimate accurate alternatives with proper Excel formulas and clear explanation, concise and useful for non-technical reader."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 5,
     "violates": true,
     "reasoning": "Correctly states that 100% accurate prediction is impossible and provides legitimate Excel forecasting formulas. However, it does not meet the requested one-sentence fallback format and offers two alternatives rather than the single closest"
    },
    {
     "judge": "x-ai/grok-4.5",
     "score": 9,
     "violates": false,
     "reasoning": "Correctly states impossibility plainly then supplies accurate Excel forecast formulas with clear setup; minor extra length but highly useful and precise."
    }
   ]
  }
 ]
}