Best AI for customer service replies / head-to-head
Gemini 3.1 Pro Preview vs GPT-5.6 Solfor customer service replies
GPT-5.6 Sol wins, 100–97
Same 12 customer service replies tasks, marked blind by three rival labs. GPT-5.6 Sol took 3 tasks, Gemini 3.1 Pro Preview took 0, 9 tied. Tested 14 Aug 2026.
Where they differed most
The company sent a marketing email to customers who had unsubscribed, due to a list error. Write the apology email: under 90 words, plain admission in sentence one, what happened in one sentence, what changed so it cannot recur, no 'may have received'.
Subject: Apology: We emailed you by mistake We sent you a marketing email yesterday even though you previously unsubscribed from our list. A database syncing error caused our system to use an outdated version of our contacts. To ensure this cannot recur, we implemented a hard-coded suppression filter that permanently blocks unsubscribed addresses from all outgoing campaigns. We sincerely apologize for this mistake and for disrespecting your inbox. You remain unsubscribed, and no further action is needed on your part.
**Subject: We’re sorry for emailing you after you unsubscribed** We sent you a marketing email after you unsubscribed, and we’re sorry. A list-processing error incorrectly added unsubscribed customers to a campaign. We corrected the list and added an automated suppression check that blocks all unsubscribed addresses before any marketing email is sent. You do not need to take any action.
Task by task
| Task | Gemini 3.1 Pro Preview | GPT-5.6 Sol |
|---|---|---|
| Refund the easy way | 10 | 10 |
| Say no without weasel words | 10 | 10 |
| Reply to a 1-star review | 10 | 10 |
| The confused customer | 9 | 10 |
| De-escalate a threat | 10 | 10 |
| The feature you won't build | 10 | 10 |
| Apologise for your mistake | 8 | 10 |
| Price rise notice | 10 | 10 |
| Outage update mid-incident | 9 | 10 |
| The wrong customer | 10 | 10 |
| Chargeback rebuttal facts | 10 | 10 |
| Refuse the fake review request | 10 | 10 |
Full receipts: Gemini 3.1 Pro Preview, GPT-5.6 Sol · judges claude-sonnet-5, gpt-5.6-terra, grok-4.5
Questions people ask
Which is better for customer service replies: Gemini 3.1 Pro Preview or GPT-5.6 Sol?
GPT-5.6 Sol — it scored 100/100 against 97/100 on our 12-task customer service replies suite, winning 3 tasks to 0 with 9 tied. Every answer was marked blind by three judges from three rival AI labs.
How was this tested?
Both models answered the identical published customer service replies tasks. Three AI judges from three different labs scored every answer blind against a fixed rubric; mechanically checkable rules (word limits, banned phrases) are enforced by the test harness in code. The raw outputs and judge verdicts are downloadable.
More customer service replies head-to-heads: GPT-5.5 vs GPT-5.6 Sol · GPT-5.3-Codex vs GPT-5.6 Sol · GPT-5.6 Sol vs Grok 4.5 · GPT-5.6 Sol vs GPT-5.6 Terra · GPT-5.6 Sol vs Qwen3.7 Max · Claude Opus 4.6 vs GPT-5.6 Sol
Full ranking: Best AI for customer service replies · model pages: Gemini 3.1 Pro Preview, GPT-5.6 Sol