Library answerAs of

Find strong reasoning that led to poor outcomes

S2reasoning outcome analysis

Strong reasoning sometimes led to poor outcomes, but it was not common. Across 1,494 matched BUY/SELL decisions, strong reasoning had 81 poor follow-up outcomes versus 75 good and 99 flat. The median follow-up change for strong-reasoning decisions was -0.67%, while weak-reasoning decisions had a median of 0%. Examples include OpenAI GPT-5’s ACN SELL on 2026-07-02 (judge score 75), which was later followed by a -38.3% same-symbol move, and Google Gemini 3.5 Flash’s APP BUY on 2026-07-24 (judge score 68), followed by -22.76%. GPT-5’s ACN SELL on 2026-07-06 scored 78 but later mapped to -32.03% and its 2026-07-19 ACN SELL scored 74 with -26.31% later change. Another strong case, APP BUY on 2026-07-25, scored 71 and later saw -21.41%.

What the data suggests: The data suggests that high judge scores do not guarantee favorable subsequent price moves; even coherent, well-grounded reasoning can precede poor outcomes. However, the pattern is modest, not decisive: strong-reasoning poor outcomes are only slightly more common than strong good outcomes, and the benchmark is based on a single matching framework with judge scores that themselves show spread/disagreement. The follow-up change is also tied to the next same-symbol decision, so timing and symbol-specific volatility matter.

The Research Library is an educational research tool. Every answer describes recorded model behavior in the benchmark and cites the records it was built from; nothing here is personalized guidance on what to do with your money, or an instruction to buy or sell any security.