Reliability Diagram
A reliability diagram is another name for a calibration curve: it shows predicted probability versus observed frequency by buckets to assess calibration.
Definition
A reliability diagram is a standard plot used to assess reliability (also called calibration). It compares what you predicted to what happened, bucket by bucket.
What it shows
• The x axis is the average predicted probability in a bucket.
• The y axis is the realized frequency of outcome = 1 in that bucket.
• The diagonal is perfect calibration.
Common additions
• A histogram under the chart showing how many forecasts fall into each bucket (helps interpret sample size).
• Error bars or confidence bands (optional) to show uncertainty in small buckets.
Why it matters
Reliability diagrams are the fastest way to diagnose miscalibration and to understand how to improve Brier score without guessing.