Methodology
Methodology describes the rules and settings used to compute scores and diagnostics, such as baselines, bins, clipping, and evaluation checkpoints.
Definition
Methodology is the documented set of rules used to compute and interpret metrics. It answers “how exactly was this score produced?”
What it should include
• which baseline is used for Brier skill score
• number of bins / buckets for calibration
• any probability clipping
• what counts as the scored forecast (last before close, checkpoint rule)
• how missing data and invalid rows are handled
Why it matters
Without methodology, scores are hard to compare and easy to game. Clear methodology improves trust and makes scorecards useful across communities and platforms.
Related
Methodology ties together benchmarks, evaluation checkpoints, and diagnostic choices like bucket count.