Which risks did contestants most often ignore?
Contestants most often ignored concentration risk. The benchmark records 603 ignored-risk mentions across 9 models, far above valuation at 180 mentions and stop/exit discipline at 155. The next most common misses were momentum/technical signals at 106, earnings/guidance at 84, and liquidity/cash at 57. The judge notes repeatedly describe oversized or repeated adds to single names, especially without explicit sizing or beta-aware controls. Valuation errors were also common, including buying expensive names while citing growth. Stop/exit discipline and momentum were frequent blind spots, but less so than concentration.
What the data suggests: The data suggests concentration and process discipline were the most persistent weak spots across the evaluated runs, with valuation a clear secondary issue. This is aggregate evidence across 367 evaluations, not a ranking of every model’s failures, and the examples span different seasons, styles, and judge passes. Some models had better risk awareness scores than others, but judge spread and short-horizon runs mean the exact importance of each risk type can vary by case.
- Judge evaluation · OpenAI GPT-4 Turbo · 2025-03-31
judge:2025-03-…amental-sage - Judge evaluation · OpenAI GPT-4 · 2025-03-31
judge:2025-03-…-news-trader - Judge evaluation · OpenAI GPT-3.5 · 2025-03-31
judge:2025-03-…nd-navigator - Judge evaluation · Anthropic Claude Sonnet 4.6 · 2026-07-02
judge:2026-07-…ropic-claude