Backtest
A backtest evaluates a forecasting method on historical data. Good backtests avoid look-ahead bias and report performance out of sample.
Definition
A backtest is a retrospective evaluation of a forecasting approach on historical events. You simulate making predictions using only information that would have been available at the time.
Why it matters
Backtests help estimate what performance you might expect going forward, but only if the test is clean. Bad backtests are often polluted by look ahead bias or data leakage.
Best practices
• Use time based splits and evaluate out of sample.
• Lock the benchmark and settings.
• Report N, coverage, and horizon mix.
Related
Backtests are often reported with Brier score and Brier skill score against a benchmark.