Calibration Error
Calibration error measures how far predicted probabilities deviate from observed frequencies across probability bins.
Definition
Calibration error summarizes miscalibration by measuring gaps between predicted probabilities and observed frequencies across probability bins. It is often visualized with a reliability diagram.
Why it matters
Calibration error makes miscalibration visible and measurable. It helps diagnose overconfidence and underconfidence patterns.
Common pitfalls
Too few forecasts: Bin statistics are unstable on small samples.
Bad binning: Wide bins can hide problems, narrow bins can amplify noise.
Learn more
For calibration tables and diagrams, see BrierScore.com.