Reliability
Reliability is another name for calibration. It describes whether forecast probabilities match observed frequencies within probability buckets.
Definition
Reliability is commonly used as a synonym for calibration. It asks whether probabilities mean what they claim.
Example
If you issue many forecasts at 80%, reliability means about 80% of those events happen.
Why it matters
Reliable forecasts let users make consistent decisions. In prediction markets, reliability helps interpret prices as probabilities and compare forecasters on equal footing.
Reliability in decomposition
In the standard Brier decomposition, the reliability term measures the penalty for miscalibration. Better reliability (closer match between predicted and observed frequencies) improves Brier score.
Related
See calibration for how reliability is evaluated with bins and calibration curves. For the complementary concept that measures differentiation, see resolution.