Overfitting (curve fitting)
Overfitting, or curve fitting, is what happens when a trading rule is tuned so closely to one historical sample that it captures the sample's accidental patterns along with any repeatable structure, so its backtested result overstates what the rule would do on new data. The more parameters and the more configurations tried on the same data, the easier it is to produce.
Senzoukria · Glossary · Updated September 2026
How it is produced
Every historical sample contains patterns that will not recur. A rule with enough adjustable pieces can be made to fit them: an extra filter that removes the worst losing day, a threshold nudged to catch one more winner, a session cutoff chosen after looking at the equity curve. Each adjustment improves the in-sample result and adds nothing the future will honour.
Selection does the same job without changing the rule. Run many configurations on one sample and keep the best; the best of many random draws looks like an edge on pure noise. This is why the number of configurations tried belongs in the record next to the result.
Signs to look for
- A steady in-sample climb followed by a flat or falling out-of-sample curve.
- Neighbouring parameter values that perform much worse than the chosen one; a sound rule usually sits on a plateau, not a spike.
- A result that collapses when costs are raised by a small amount.
- Rules with conditions that have no stated economic or microstructure reason, added after inspecting losses.
- A strong result obtained after a large number of variants were tried and discarded.
Checks that expose it
| Check | Question it answers |
|---|---|
| Walk-forward | Do parameters chosen on the past hold on the period after it? |
| Parameter stability | Does the rule still work one step away on each axis? |
| Deflated Sharpe ratio | Is the result distinguishable from the best of the configurations tried? |
| Probability of backtest overfitting | How often does the in-sample winner underperform out of sample? |
| Permutation test | Is the result distinguishable from chance on shuffled data? |
| Cost sensitivity | Does the edge survive higher fees and slippage? |
In Senzoukria
The Automatic backtest panel's Gauntlet section, titled Gauntlet — is this edge real?, runs the checks above on a finished walk-forward: a Deflated Sharpe corrected for the number of configurations tried, a permutation test against pure chance, and the Probability of backtest overfitting, built from a full-period grid in which every configuration is rerun on all bars. Each criterion is marked eliminatory or advisory and the section resolves to GO, NO-GO or CONDITIONAL.
The Deflated Sharpe reading is worded in three levels: the result survives the correction; the number of configurations tried explains a good part of it; or it is not distinguishable from the best of a random draw, which the panel describes as what an overfit looks like. A parameter stability chart draws the chosen configuration against its immediate neighbours on the grid and explains that a plateau is what a real edge looks like. When the sweep grows too large the panel warns that past that point you are measuring the luck of an overfit. These are reports on the supplied data, not a guarantee.
Common mistakes
- Adding a filter because it removes a specific losing streak.
- Optimising on the whole history and then reporting that history as the result.
- Confusing a narrow spike in the parameter grid with a discovery.
- Discarding the count of variants tried once a good one is found.
Related
- Backtest
- How to test a trading strategy
- Futures backtesting guide
- Parameter sweep
- In-sample vs out-of-sample
This page in other languages
Frequently asked questions
- How many parameters are too many?
- There is no fixed number; what matters is the number of adjustable pieces relative to the number of independent trades in the sample, and how many combinations were tried. A rule with two parameters swept over hundreds of combinations on a few dozen trades can be as overfitted as a rule with ten parameters. Record the count of configurations and use a deflated statistic that accounts for it.
- Is a walk-forward result immune to overfitting?
- No. Walk-forward moves selection out of the test windows, which removes the most direct form of curve fitting, but the analyst can still overfit the procedure itself by changing slices, grids or selection rules after seeing the outcome. Fix those choices before the run and record every run, including the ones you discard.
- What does a parameter plateau tell me?
- It tells you the rule's result does not depend on landing on one exact value. If neighbouring configurations, one step away on each axis, perform nearly as well, the rule is capturing something broader than a coincidence of the sample. A single spike surrounded by poor neighbours is the opposite signal.