Backtest overfitting
Did the strategy discover a recurring relationship, or memorize one convenient history?
No single statistic answers that. Look for an edge that survives changed samples, chronology, assumptions, and trade order without quietly changing the rules.
THE PRESSURE TEST
Move the convenient assumptions.
Apply several independent stresses. A strategy that survives only one favorable path or one tuned window carries narrow evidence even when its headline curve is clean.
- Compare chronological halves without retuning the strategy.
- Remove the largest winners and measure how much of the claim remains.
- Resample trade outcomes to expose dependence on one lucky sequence.
- Increase execution friction without changing the original trade list.
A 300-trade record stays positive in aggregate, but the first half is flat, the second half contains almost the entire result, and five trades explain most of the net outcome. That does not prove the strategy is false. It does show that the broad historical curve carries less evidence than it appears to.
The check stops here.
Passing these checks does not prove the strategy will persist. Parameter-search history, abandoned variants, market impact, source-code behavior, and future regime changes can remain outside the submitted record.
SEE THE CALCULATION, NOT A CLAIM