Best AI for customer service replies / head-to-head
Claude Sonnet 5 vs Gemini 3.1 Pro Previewfor customer service replies
Gemini 3.1 Pro Preview wins, 97–93
Same 12 customer service replies tasks, marked blind by three rival labs. Gemini 3.1 Pro Preview took 4 tasks, Claude Sonnet 5 took 2, 6 tied. Tested 14 Aug 2026.
Where they differed most
A customer wants a refund on a personalised item after 60 days; policy is 30 days and the item was made to order. Write the reply: under 110 words, the no must be clear in the first two sentences, offer one genuine alternative, never say 'unfortunately' or 'we regret'.
Subject: Your Refund Request Hi [Name], Thank you for reaching out. I'm not able to approve a refund for this item—as a personalised, made-to-order piece, it falls outside our 30-day policy, and 60 days have now passed. I do want to help find a solution. If there's a specific issue with the item (fit, damage, error in personalisation), please let me know and I'll look into options like a repair or partial credit toward a new order. If it's simply no longer needed, I understand that's frustrating, but the custom nature of the piece means we…
Thank you for reaching out about your personalized order. We cannot process a refund for this made-to-order item, as your request falls outside our 30-day return window. Because the product was custom-made specifically for you, it cannot be restocked or resold. As an alternative, I can offer you a 30% discount on any future purchase from our store. Please reply to this email if you would like me to activate this discount code for you. Best regards, Customer Support
Task by task
| Task | Claude Sonnet 5 | Gemini 3.1 Pro Preview |
|---|---|---|
| Refund the easy way | 10 | 10 |
| Say no without weasel words | 5 | 10 |
| Reply to a 1-star review | 10 | 10 |
| The confused customer | 10 | 9 |
| De-escalate a threat | 10 | 10 |
| The feature you won't build | 10 | 10 |
| Apologise for your mistake | 10 | 8 |
| Price rise notice | 9 | 10 |
| Outage update mid-incident | 8 | 9 |
| The wrong customer | 9 | 10 |
| Chargeback rebuttal facts | 10 | 10 |
| Refuse the fake review request | 10 | 10 |
Full receipts: Claude Sonnet 5, Gemini 3.1 Pro Preview · judges gemini-3.1-pro-preview, gpt-5.6-terra, grok-4.5
Questions people ask
Which is better for customer service replies: Claude Sonnet 5 or Gemini 3.1 Pro Preview?
Gemini 3.1 Pro Preview — it scored 97/100 against 93/100 on our 12-task customer service replies suite, winning 4 tasks to 2 with 6 tied. Every answer was marked blind by three judges from three rival AI labs.
How was this tested?
Both models answered the identical published customer service replies tasks. Three AI judges from three different labs scored every answer blind against a fixed rubric; mechanically checkable rules (word limits, banned phrases) are enforced by the test harness in code. The raw outputs and judge verdicts are downloadable.
More customer service replies head-to-heads: Gemini 3.1 Pro Preview vs GPT-5.6 Sol · Claude Sonnet 5 vs GPT-5.6 Sol · Gemini 3.1 Pro Preview vs GPT-5.5 · Claude Sonnet 5 vs GPT-5.5 · Gemini 3.1 Pro Preview vs GPT-5.3-Codex · Claude Sonnet 5 vs GPT-5.3-Codex
Full ranking: Best AI for customer service replies · model pages: Claude Sonnet 5, Gemini 3.1 Pro Preview