What is Model Calibration?
How well a model's stated confidence matches how often it is actually right.
Formal Definition
Calibration measures the alignment between a model's expressed confidence and its real accuracy: a well-calibrated model is right about 70% of the time when it claims 70% confidence. In finance this connects confidence to risk. AI Stock Challenge's rubric rewards uncertainty discipline, favoring models that hedge when the evidence is thin instead of overclaiming.
In Simple Terms
It is whether an AI knows what it does not know. A well-calibrated model is confident when it should be and cautious when the evidence is weak, instead of sounding equally sure about everything.
Example
A model that expresses low conviction on a murky, low-data stock and high conviction on a clear one is better calibrated than a model that sounds certain about both.