Library answerAs of

Where did the judge panel disagree most?

S2judge analysis

The biggest judge disagreement in the packet was on xAI Grok 4.3. The widest single-record spread was 45 points on 2026-08-01, where OpenAI scored it 60, Anthropic 42, and xAI 87. Other large splits for the same model were 42 points on 2026-08-27 and 40 points on 2026-08-21. Across all 315 panel-scored evaluations, the overall composite range across judges ran from 2 to 45, with a median of 21. By model, xAI Grok 4.3 had the highest median judge spread at 28 over 65 evaluations, above Google Gemini 3.1 Pro at 21 over 62, Anthropic Claude Sonnet 4.6 at 20 over 63, Google Gemini 3.5 Flash at 20 over 62, OpenAI GPT-5 at 15 over 60, and OpenAI GPT-6 Astra at 5 over 3.

What the data suggests: The data suggests judges disagreed most on xAI Grok 4.3, especially in late-summer evaluations where the xAI judge gave much higher scores than OpenAI and Anthropic. The pattern points to disagreement over reasoning quality, risk control, and evidence grounding rather than over final outcomes alone. Caveats: this is a benchmark sample, not a full market study; the spread measures panel disagreement on scored evaluations; and the highest-spread model also appears multiple times, so one model’s repeated runs may dominate the top end.

The Research Library is an educational research tool. Every answer describes recorded model behavior in the benchmark and cites the records it was built from; nothing here is personalized guidance on what to do with your money, or an instruction to buy or sell any security.