Data Dictionary — AI Financial Reasoning Dataset

Model decision

source: portfolio/<model_id> → tradeHistory[]

cadence: One record per BUY / SELL / HOLD the model emitted, every decision day

The core longitudinal record: what the model decided, at what price, and its written rationale at the time. HOLD records carry the same reasoning fields as trades, so the dataset captures decisions not to act as well as actions.

Field	Type	Description
`season`	integer	1 or 2. Added on export.
`model_id`	string	Stable internal model id, e.g. s2-anthropic-claude, gpt-trend-navigator. Added on export.
`model_name`	string	Display name at time of export, e.g. "Anthropic Claude Sonnet 4.6". Added on export.
`provider`	string	openai · anthropic · xai · google. Added on export.
`type`	enum	BUY, SELL or HOLD.
`symbol`	string	Ticker the decision concerns (S&P 500 universe).
`shares`	number	Shares traded. 0 for HOLD and for BUYs the model flagged but could not fund.
`price`	number	Price the decision was evaluated / filled at (USD).
`timestamp`	ISO 8601	When the decision was recorded (UTC).
`totalValue`	number	shares × price (USD).
`cashEffect`	number	Signed cash impact of a BUY (negative) or SELL (positive). Not present on HOLD.
`currentPrice`	number	Price observed for a HOLD. Not present on BUY / SELL in Season 2.
`reason`	text	The model’s full written rationale for this decision: cited metrics, thesis, risks, triggers. Typically 300–1,500 characters.
`analysis`	text	One-line summary of the decision written by the model.
`contextual_memory`	text	Notes the model wrote for its future self (what to watch, when to re-enter, conditions to reassess). Often empty in Season 1.
`description`	text	Mechanical summary, e.g. "Sold 102 ANET at 162.2".

Portfolio state

source: portfolio/<model_id> (top level)

cadence: One document per model, updated every trading day; performanceHistory is the daily value series

The state each decision was made against and what it produced: holdings with cost basis, a daily total-value series from the model’s first day, the decision latency series (Season 2), and the model’s configuration.

Field	Type	Description
`modelInfo.id / name / description`	object	Model identity and, for Season 1, the strategy brief it was given; for Season 2, the shared-prompt note.
`modelInfo.strategy`	string	Season 1: the model-specific strategy. Season 2: "Shared financial-reasoning prompt; the model is the only variable".
`modelInfo.inception`	date	First day of the portfolio.
`modelInfo.initialCapital`	number	100000 for every model.
`holdings[]`	array	Current positions: symbol, shares, currentPrice, averageCost, sector, industry. Cash is the $CASH row.
`performanceHistory[]`	array	{ date, value }: total portfolio value at the end of each day. This is the outcome series a decision can be joined to.
`latencyHistory[]`	array	{ date, seconds }: wall-clock time the model took to produce that day’s decisions. Season 2 only.
`recentActions[]`	array	The last few decision records (same shape as Model decision).
`currentValue`	number	Latest total value (USD).
`LastTradeDay`	date	Last day the model traded.

Daily leaderboard snapshot

source: leaderboards/<YYYY-MM-DD>-leaderboardData.json

cadence: One document per day the leaderboard was updated

Cross-model standings as they were on each day: portfolio value, gains, trade counts and last activity. Useful for reconstructing what rank a model held when it made a given decision (the models were told their rank).

Field	Type	Description
`snapshot_date`	date	Date of the snapshot. Added on export.
`season`	string	season-1 / season-2. Present from Season 2 on.
`lastUpdated`	ISO 8601	When the snapshot was written.
`models[].modelId`	string	Model id. Present from Season 2 on; Season 1 rows carry modelName only.
`models[].modelName`	string	Display name at the time.
`models[].portfolioValue`	number	Total value (USD).
`models[].dayGainPercent`	number	Day-over-day change (%).
`models[].totalGainPercent / totalGainDollars`	number	Change since inception.
`models[].totalTrades`	number	Cumulative decision count.
`models[].recentActivity`	enum	BUY / SELL / HOLD: the most recent decision type.

Judge evaluation

source: marketdata/assessments/<as_of_date>-<model_id>-assessment.json

cadence: Season 1: monthly per model, single judge. Season 2: every trading day per model, anonymized three-judge panel

An LLM-as-judge audit of the model’s whole decision horizon up to as_of_date: headline scores, eleven rubric dimensions, a claim ledger checking the model’s stated facts against the market snapshot, and structured analyses of strategy fit, risk and process. Season 2 documents include each judge’s independent scores and their agreement.

Field	Type	Description
`score_version`	string	Rubric version, e.g. grounded_financial_reasoning_judge_v0.4.
`model_id / model_name`	string	Model evaluated.
`as_of_date`	date	Horizon end for this evaluation.
`decision_timestamp / market_snapshot_timestamp`	string	Last decision judged and the market snapshot the claims were checked against.
`lookahead_check`	object	{ passed, issue }: whether the judge confirmed no post-decision data leaked into the assessment.
`sample_size_status`	enum	How many decisions the score rests on (e.g. strong).
`scores.reasoning_quality_score_0_100`	number	Quality of the reasoning.
`scores.evidence_grounding_score_0_100`	number	How well claims were supported by the data the model had.
`scores.outcome_score_0_100`	number	Realized outcomes of the decisions.
`scores.data_reliability_score_0_100`	number	Reliability of the data the model cited.
`scores.final_financial_reasoning_score_0_100`	number	Headline reasoning score (panel median in Season 2).
`scores.median_reasoning_seconds`	number	Median decision latency over the horizon. Season 2 only.
`scores.reasoning_efficiency_score_0_100`	number	Reasoning quality per second of thinking. Season 2 only.
`scores.composite_index_0_100`	number	90% reasoning + 10% efficiency; the site’s Total Score. Season 2 only.
`dimension_scores.*`	number ×11	action_rationale_alignment, thesis_quality, strategy_fit, risk_awareness, portfolio_discipline, temporal_consistency, decision_update_quality, uncertainty_discipline, claim_grounding, metric_correctness, data_consistency.
`claim_ledger[]`	array	{ claim, symbol, claim_type, support_status, market_data_used[], judge_note }: individual factual claims the model made, and whether the data supported them.
`strategy_fit_analysis`	object	declared_strategy, evidence_used_by_model[], evidence_expected_for_strategy[], fit_label, notes.
`risk_analysis`	object	risks_identified_by_model[], risks_visible_in_market_data_but_ignored[], concentration_or_sizing_risks[], risk_score_note.
`portfolio_process_analysis`	object	cash_discipline, position_sizing, thesis_continuity, update_quality (free text).
`major_strengths[] / major_weaknesses[]`	array	Judge’s bullet findings.
`data_quality_flags[] / missing_instrumentation[]`	array	Caveats about the record the judge was given.
`public_summary`	object	one_sentence, badge, best_supported_claim, weakest_claim, most_important_risk_ignored, what_would_improve_score[].
`panel`	object	Season 2 only. method, anonymized, num_judges, judges[] (judge_id, provider, model, family, scores, badge), agreement (composite min / median / max / range / stdev).
`decision_summary`	object	total_decisions, trading_days, by_type, first_decision_date, last_decision_date over the horizon.
`judged_at`	ISO 8601	When the evaluation was produced.

Evaluation time series

source: marketdata/assessments/<season>-timeseries.json

cadence: One document per season, one point per model per as_of_date

Compact per-model series of headline and dimension scores over time, for plotting drift, learning or degradation without loading every full evaluation.

Field	Type	Description
`season / score_version / generated_at`	string	Series metadata.
`as_of_dates[]`	array	Every evaluation date in the season.
`models.<id>.model_name`	string	Display name.
`models.<id>.series[]`	array	{ date, decisions, scores{…}, dimension_scores{…}, badge } per as_of_date.

Not included

The daily AI market analyses published on the site (market-analysis pages). They are site content, not part of the dataset.
Vendor market-data snapshots (daily quotes, fundamentals, news feeds) that the models were shown. AIStockChallenge is not licensed to redistribute them. Prices at decision time and the metrics a model cited survive in the AIStockChallenge-generated records (decision price, judge claim ledger market_data_used).
Raw prompts and provider API payloads. The shared Season 2 prompt is documented on the methodology page.

record shapes verified against live storage · counts as of 2026-09-14