{
 "entity": "glm-5-2",
 "entity_type": "model",
 "category": "Spreadsheets",
 "suite": "suite-2026-08-tasks/spreadsheets",
 "model": "z-ai/glm-5.2",
 "judges": [
  "anthropic/claude-sonnet-5",
  "google/gemini-3.1-pro-preview",
  "openai/gpt-5.6-terra"
 ],
 "judge_protocol": "judge-2026-08",
 "anchors": "solo-2026-08-a",
 "score": 96,
 "caps": 0,
 "cost_usd": 0.151,
 "started_at": "2026-08-13T08:51:36.332Z",
 "finished_at": "2026-08-13T08:54:49.753Z",
 "tasks": [
  {
   "id": "sp1",
   "name": "Excel formula from a description",
   "prompt": "In Excel, column A holds order dates and column B holds order values. Write a single formula for cell D1 that totals only the orders placed in the last full calendar month, relative to today's date. Give the formula and one sentence explaining it. Do not use a pivot table or a helper column.",
   "output": "`=SUMIFS(B:B, A:A, \">=\"&EOMONTH(TODAY(),-2)+1, A:A, \"<=\"&EOMONTH(TODAY(),-1))`\n\nThis formula uses the SUMIFS function to add up the values in column B for any dates in column A that fall between the first and last day of the previous calendar month, calculated dynamically using the TODAY and EOMONTH functions.",
   "latency_ms": 6034,
   "cost": 0.00161088,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Formula is correct, no helper column/pivot, single formula, clear explanation. Minor stylistic nitpicks only."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response provides a perfectly accurate Excel formula to sum the previous month's values, accompanied by exactly one clear explanatory sentence, following all constraints flawlessly."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "The formula correctly sums column B where column A dates fall from the first through the last day of the previous full calendar month, using dynamic TODAY/EOMONTH boundaries. It provides one formula and one clear explanatory sentence."
    }
   ]
  },
  {
   "id": "sp2",
   "name": "Fix a broken formula",
   "prompt": "This Excel formula returns #VALUE! : =SUMIF(A:A,\">\"&TODAY()-30,B:B)+VLOOKUP(C1,Sheet2!A:D,5,FALSE). Identify every fault and give the corrected formula. Sheet2 has columns A to D only.",
   "output": "Here are the faults identified in the formula, followed by the corrected version:\n\n### Faults Identified:\n1. **Column Index Out of Bounds:** The `VLOOKUP` function specifies a column index of `5` (`col_index_num`), but the table array `Sheet2!A:D` only contains 4 columns (A, B, C, D). This will cause a `#REF!` error. The maximum column index you can use for this range is `4`.\n2. **Data Type Mismatch (Causes #VALUE!):** Using the `+` operator to add the `SUMIF` result and the `VLOOKUP` result will return a `#VALUE!` error if the `VLOOKUP` returns text instead of a number. If you correct the column index to 4, but column D contains text, the math operation will fail. \n3. **Missing Error Handling:** If the value in cell `C1` is not found in `Sheet2!A:A`, the `VLOOKUP` will return an `#N/A` error, which will also break the addition. It needs to be wrapped in an `IFERROR` function to default to 0 (or another number) if no match is found.\n4. **Potential Error Propagation in SUMIF:** If there is an existing `#VALUE!` error anywhere in column B (the sum range), the `SUMIF` function will propagate that error instead of ignoring it. \n\n### Corrected Formula:\nAssuming you want to pull the value from the 4th column (Column D) and safely add it to your `SUMIF` result:\n\n`=SUMIF(A:A,\">\"&TODAY()-30,B:B) + IFERROR(N(VLOOKUP(C1,Sheet2!A:D,4,FALSE)),0)`\n\n### Why this works:\n* **`4` instead of `5`:** Fixes the out-of-bounds column reference so it correctly looks at Column D.\n* **`IFERROR(..., 0)`:** Ensures that if `C1` is not found on Sheet2, it returns `0` instead of `#N/A`, allowing the math to continue.\n* **`N(...)`:** The `N()` function converts text to a number (text becomes 0, numbers stay the same). This prevents the `#VALUE!` error if the `VLOOKUP` happens to pull a text string from Sheet2. \n\n*(Note: If you prefer a formula that ignores text and errors entirely without explicitly adding them, you can also use `=SUM(SUMIF(A:A,\">\"&TODAY()-30,B:B), IFERROR(VLOOKUP(C1,Sheet2!A:D,4,FALSE),0))`, as the `SUM` function automatically ignores text.)*",
   "latency_ms": 181650,
   "cost": 0.0075933,
   "task_score": 7,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 7,
     "violates": false,
     "reasoning": "Correctly identifies col_index=5 as the real cause (though mislabels as #REF! rather than #VALUE!, a minor inaccuracy) and gives sound corrected formula with IFERROR/N(). Clear, useful, slightly verbose."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response accurately identifies the out-of-bounds VLOOKUP index and correctly explains the potential causes for a #VALUE! error. The corrected formula is robust, handling errors and text values effectively."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 6,
     "violates": false,
     "reasoning": "Correctly fixes the invalid VLOOKUP column index. However, index 5 causes #REF!, not #VALUE!, and the alleged text, lookup-miss, and SUMIF-error faults are conditional rather than established. The added error handling may undesirably mask d"
    }
   ]
  },
  {
   "id": "sp3",
   "name": "Clean messy data",
   "prompt": "These postcodes were typed by hand: 'sw1a1aa', 'SW1A 1AA', ' m1 4wb', 'M14WB', 'ec1v9nr'. Give a single Excel formula that normalises any one of them to standard UK format (uppercase, one space before the final three characters). Assume the raw value is in A2.",
   "output": "Here is a single Excel formula that will normalise the postcode in cell A2 to the standard UK format:\n\n`=UPPER(REPLACE(SUBSTITUTE(A2,\" \",\"\"),LEN(SUBSTITUTE(A2,\" \",\"\"))-2,0,\" \"))`\n\n**How it works:**\n1. `SUBSTITUTE(A2,\" \",\"\")` removes all existing spaces from the postcode.\n2. `LEN(...)-2` calculates the position of the 4th character from the end (which is where the space should be inserted).\n3. `REPLACE(..., ..., 0, \" \")` inserts a single space at that exact position without deleting any characters (because `num_chars` is set to `0`).\n4. `UPPER(...)` converts the entire resulting string to uppercase.",
   "latency_ms": 18382,
   "cost": 0.0077512,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Formula correctly strips spaces, inserts space before last 3 chars, uppercases; works for all examples. Minor: doesn't trim leading space in ' m1 4wb' before removing internal spaces, but SUBSTITUTE removes all spaces so fine."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The formula is perfectly accurate, elegantly handles the spacing logic using REPLACE and SUBSTITUTE, and is explained clearly and concisely for a non-technical user."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Formula correctly removes spaces, inserts one space before the final three characters, and converts to uppercase for all listed inputs. It directly uses A2 and is clear; extra explanation is accurate and useful."
    }
   ]
  },
  {
   "id": "sp4",
   "name": "Choose the right chart",
   "prompt": "A shop has 14 months of weekly revenue and wants to see whether sales are seasonal. In under 90 words, say which chart to build, which fields go on which axis, and one thing this chart will NOT tell them.",
   "output": "Build a **line chart**. Place the \"Week/Date\" field on the **X-axis** and \"Revenue\" on the **Y-axis**. This will visually highlight repeating peaks and troughs across the 14 months, making seasonal patterns easy to spot. \n\nHowever, this chart will NOT tell them the **underlying cause** of these fluctuations (e.g., whether a revenue spike is due to a specific holiday, a successful marketing campaign, or an external economic factor).",
   "latency_ms": 5774,
   "cost": 0.0031406,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Correct, clear, meets word limit, covers chart type, axes, and limitation concisely."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response perfectly follows all instructions, accurately recommending a line chart with correct axis mappings and a valid limitation, all well under the 90-word limit."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Correctly recommends a line chart with Week/Date on X-axis and Revenue on Y-axis, explains its seasonal-pattern value, and accurately states that it cannot establish causes. It is concise, clear, and under 90 words."
    }
   ]
  },
  {
   "id": "sp5",
   "name": "Google Sheets QUERY",
   "prompt": "In Google Sheets, tab 'Sales' has headers in row 1: Date, Rep, Region, Amount. Write one QUERY() formula returning total Amount per Rep for the North region only, sorted highest first. Return only the formula.",
   "output": "=QUERY('Sales'!A1:D, \"SELECT B, SUM(D) WHERE C = 'North' GROUP BY B ORDER BY SUM(D) DESC\", 1)",
   "latency_ms": 2082,
   "cost": 0.00103296,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Correct QUERY syntax, aggregates by Rep, filters North, sorts descending, includes header option. Minor: label ordering could differ, but functionally correct and concise."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response provides exactly the requested formula, which correctly filters, groups, and sorts the data using the appropriate columns and syntax."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Correct Google Sheets QUERY syntax: filters North, groups by Rep, sums Amount, sorts totals descending, and specifies one header row. Returns only the requested formula."
    }
   ]
  },
  {
   "id": "sp6",
   "name": "Spot the error in the numbers",
   "prompt": "Monthly revenue: Jan 42,100; Feb 38,400; Mar 41,900; Apr 4,220; May 43,050; Jun 44,110. State in one sentence what is almost certainly wrong with this data and what the April figure most likely should be. Do not speculate about business causes.",
   "output": "The April figure is almost certainly missing a zero due to a data entry error, and should most likely be 42,200.",
   "latency_ms": 6021,
   "cost": 0.0021374,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 7,
     "violates": false,
     "reasoning": "Correctly identifies likely data entry error and plausible corrected value in one sentence, follows instructions, concise; minor quibble: 42,200 vs other estimates plausible but reasonable."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response perfectly follows all instructions, providing a clear, accurate, one-sentence explanation of the likely data entry error without speculating on business causes."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Correctly identifies the implausible April outlier as a likely missing zero and infers 42,200 from surrounding monthly values. It is one sentence, concise, and avoids speculating about business causes."
    }
   ]
  },
  {
   "id": "sp7",
   "name": "Pivot logic in words",
   "prompt": "Explain to a non-technical manager, in exactly 3 bullet points of no more than 20 words each, what a pivot table does. Do not use the words 'aggregate', 'dimension' or 'axis'.",
   "output": "* It quickly summarizes large amounts of raw data into clear, easy-to-read reports.\n* It lets you drag and drop columns to view information from different perspectives.\n* It automatically calculates totals, counts, and averages based on your chosen categories.",
   "latency_ms": 8048,
   "cost": 0.0037098,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Clear, accurate, concise bullets within word limits, avoids banned words, understandable for non-technical manager."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response perfectly explains pivot tables to a non-technical manager. It strictly follows the 3-bullet format, stays well under the 20-word limit per bullet, and successfully avoids all forbidden words."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Exactly three bullets, each under 20 words; banned words are absent. Accurate, clear, and manager-friendly explanation of summarizing, rearranging views, and calculating results."
    }
   ]
  },
  {
   "id": "sp8",
   "name": "Percentage change trap",
   "prompt": "A price rose from £80 to £100, then fell from £100 to £80. State both percentage changes to one decimal place and explain in one sentence why they are not the same number.",
   "output": "The percentage changes are a **25.0% increase** (from £80 to £100) and a **20.0% decrease** (from £100 to £80). \n\nThey are not the same number because the first percentage is calculated using the lower starting price of £80 as its base, while the second percentage is calculated using the higher starting price of £100 as its base.",
   "latency_ms": 4704,
   "cost": 0.001241896,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Correct percentages, one decimal place, clear one-sentence explanation of base difference. Meets all instructions concisely."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response perfectly calculates the percentage changes to one decimal place and provides a clear, accurate one-sentence explanation for why they differ, following all instructions flawlessly."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Both percentage changes are correct to one decimal place, and the explanation accurately identifies the differing bases. It fully follows the requested format and is clear and concise."
    }
   ]
  },
  {
   "id": "sp9",
   "name": "Conditional formatting rule",
   "prompt": "In Excel, give the exact custom formula for a conditional formatting rule applied to A2:A500 that highlights a cell when its value is more than 20% below the average of that whole range. Return the formula and nothing else.",
   "output": "=A2 < AVERAGE($A$2:$A$500) * 0.8",
   "latency_ms": 2750,
   "cost": 0.00114625,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Correct, exact formula matching required condition, follows instruction to return only formula."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response provides the exact correct formula and strictly follows the negative constraint to return nothing else."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "The formula correctly compares each relative cell in A2:A500 to 80% of the fixed average for $A$2:$A$500, matching “more than 20% below.” It returns only the formula."
    }
   ]
  },
  {
   "id": "sp10",
   "name": "Reconcile two lists",
   "prompt": "You have invoice numbers in Sheet1!A:A and payments in Sheet2!A:A. Give one Excel formula for Sheet1!B2 that returns 'PAID' or 'UNPAID', and state in one sentence what it will wrongly report if an invoice number has a trailing space.",
   "output": "**Formula:**\n`=IF(ISNUMBER(MATCH(A2, Sheet2!A:A, 0)), \"PAID\", \"UNPAID\")`\n\n**Trailing Space Issue:**\nIf an invoice number has a trailing space, the formula will wrongly report \"UNPAID\" for an invoice that was actually paid because the exact match function will fail to find the corresponding payment.",
   "latency_ms": 9654,
   "cost": 0.00366294,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Correct formula and accurate explanation of trailing space issue; concise and meets task requirements well, minor room for edge-case mention."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response provides a correct Excel formula and accurately explains the trailing space issue in exactly one sentence, perfectly meeting all instructions."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "The formula correctly checks whether the Sheet1 invoice number appears in Sheet2 and returns the required statuses. The trailing-space explanation is accurate, concise, and directly answers the question."
    }
   ]
  },
  {
   "id": "sp11",
   "name": "Explain a formula back",
   "prompt": "Explain in under 60 words what this does: =IFERROR(INDEX(B:B,MATCH(1,(C:C=\"UK\")*(D:D>1000),0)),\"none\"). Mention one reason it might be slow on a large sheet.",
   "output": "This formula returns the first value from column B where column C is \"UK\" and column D exceeds 1000. If no match exists, it returns \"none\". It runs slowly on large sheets because it uses full-column references (e.g., C:C) with array calculations, forcing Excel to evaluate over a million rows per condition.",
   "latency_ms": 8228,
   "cost": 0.0030464,
   "task_score": 10,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 9,
     "violates": false,
     "reasoning": "Accurate, concise explanation within word limit, correctly identifies slowness cause (full-column array references)."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response perfectly explains the formula's function and accurately identifies full-column array calculations as the cause of slowness. It is clear, concise, and strictly adheres to the word count constraint."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 10,
     "violates": false,
     "reasoning": "Accurately explains the first matching value, fallback result, and a valid performance concern: full-column array calculations evaluate very large ranges. It is concise, clear, and under 60 words."
    }
   ]
  },
  {
   "id": "sp12",
   "name": "Refuse the impossible",
   "prompt": "Write an Excel formula that predicts next month's revenue with 100% accuracy from the previous 12 months. If this is not possible, say so plainly in one sentence and give the closest legitimate alternative instead.",
   "output": "It is impossible to predict next month's revenue with 100% accuracy using only historical data due to unpredictable external business factors. \n\nThe closest legitimate alternative is to use Excel's Exponential Triple Smoothing function, which accounts for both trends and seasonality:\n\n`=FORECAST.ETS(A14, B2:B13, A2:A13, 12)`\n\n*(Note: This assumes your previous 12 months of dates are in cells A2:A13, your corresponding revenues are in B2:B13, and the target date for next month is in A14. The \"12\" at the end tells Excel to look for a 12-month seasonal pattern).*",
   "latency_ms": 8481,
   "cost": 0.001600176,
   "task_score": 8,
   "judges": [
    {
     "judge": "anthropic/claude-sonnet-5",
     "score": 8,
     "violates": false,
     "reasoning": "Correctly states impossibility, gives legitimate FORECAST.ETS alternative with clear explanation; minor issue: seasonality parameter usage slightly imprecise but overall accurate and useful."
    },
    {
     "judge": "google/gemini-3.1-pro-preview",
     "score": 10,
     "violates": false,
     "reasoning": "The response perfectly addresses the impossibility of 100% accuracy in a single sentence, then provides a highly useful and accurate alternative formula with clear, concise instructions for a non-technical user."
    },
    {
     "judge": "openai/gpt-5.6-terra",
     "score": 8,
     "violates": false,
     "reasoning": "Correctly states 100% accuracy is impossible and provides a valid Excel forecasting alternative with clear range assumptions. Minor limitation: 12 observations are generally too few to establish a reliable 12-month seasonal pattern, though "
    }
   ]
  }
 ]
}