A high Sharpe ratio can be the reward for a good strategy. It can also be the reward for asking the same history enough questions and publishing the luckiest answer.
The deflated Sharpe ratio asks the missing question: after accounting for how many alternatives were tried, how short and non-normal the return history is, does the selected backtest still look exceptional? It is a selection diagnostic, not a better-looking performance badge.
What the raw Sharpe ratio leaves out
The raw Sharpe ratio measures average excess return relative to return volatility. It does not record the research process that produced the strategy.
That omission matters when a researcher tests many entry rules, markets, timeframes, filters, and parameter combinations. Even if none contains a durable edge, some candidates will look strong by chance. Choosing the highest Sharpe and discussing it as if it were the only test hides the search that created the winner.
The raw figure also compresses the return distribution into a mean and a standard deviation. Short samples are uncertain. Skew and fat tails can make the usual statistical assumptions unreliable. A smooth-looking average can coexist with a return shape that is much less stable than the headline suggests.
This is separate from the per-trade versus per-day Sharpe basis. The basis tells you which return series was measured. Deflation asks whether the best measured result survived the way it was selected.
What the deflated Sharpe ratio corrects
The deflated Sharpe ratio estimates whether an observed Sharpe clears a higher hurdle after selection bias and non-normal returns are considered. The method was set out by David H. Bailey and Marcos López de Prado in The Deflated Sharpe Ratio.
Its inputs describe the evidence behind the headline:
| Input | Question it forces the researcher to answer |
|---|---|
| Observed Sharpe | How strong was the selected return series? |
| Number of trials | How wide was the search that produced it? |
| Variation across trials | How much performance dispersion did the search create? |
| Track length | How much independent evidence supports the estimate? |
| Skew and kurtosis | How far are returns from a simple normal shape? |
More trials raise the benchmark because a lucky winner becomes easier to find. A shorter or more irregular return history weakens confidence because the Sharpe estimate has more sampling uncertainty. Deflation does not accuse every optimisation of cheating. It prices the search into the claim.
Deflated Sharpe and PBO answer different questions
The deflated Sharpe ratio and Probability of Backtest Overfitting are related, but they are not interchangeable.
The deflated Sharpe ratio asks whether the selected performance is statistically convincing after accounting for the research search and the return distribution. PBO asks how often the selection procedure chooses an in-sample winner that disappoints out-of-sample.
Bailey, Borwein, López de Prado, and Zhu developed PBO using combinatorially symmetric cross-validation in The Probability of Backtest Overfitting. Instead of admiring one winning curve, the method repeatedly changes which chronological blocks are used for selection and evaluation. It then observes whether the in-sample winner falls below the out-of-sample middle of the pack.
| Diagnostic | Main question | Main failure exposed |
|---|---|---|
| Deflated Sharpe ratio | Is the selected Sharpe credible after the search? | Performance inflated by multiple testing and non-normal returns |
| PBO | Does the selection rule keep choosing candidates that fail out-of-sample? | An unstable model-selection process |
| Hold-out test | Does the final frozen strategy work on untouched data? | A strategy fitted to the development sample |
Using one does not make the others redundant. A frozen hold-out remains useful because it represents genuinely untouched evidence. PBO examines the wider selection machinery. Deflated Sharpe examines the statistical claim attached to the chosen result.
A selection log is the practical starting point
The calculation is only as honest as the research log behind it. If discarded trials disappear, the adjustment starts from a false search size.
A defensible workflow records:
- The strategy family and economic rationale before optimisation.
- Every market, timeframe, rule variant, filter, and parameter set tested.
- One comparable return series and Sharpe basis for every candidate.
- The selection rule before the final candidate is chosen.
- The held-out data that remains untouched until selection is complete.
- The selected result, the deflated diagnostic, the PBO evidence where appropriate, and the final out-of-sample result.
Near-identical parameter settings are correlated trials, so counting alternatives is not always as simple as counting files. That is a limitation to document, not a reason to pretend the alternatives never existed. The search space is part of the evidence.
For a prop-firm strategy, selection must also respect the rule objective. The highest Sharpe candidate is not automatically the candidate with the safest daily loss path, floating-equity drawdown, or time-to-target. The account mechanics on the funding page belong in the test specification before optimisation begins.
What this diagnostic cannot prove
A passing deflated Sharpe ratio does not prove that an edge will persist live. It does not repair look-ahead bias, missing costs, bad timestamps, survivorship bias, or a strategy that was redesigned after viewing the hold-out.
It also cannot recover an undocumented search. If a researcher remembers only the final configuration, no formula can reconstruct the rejected experiments reliably.
The diagnostic is strongest as one layer in a larger process: clean data, a predeclared selection rule, realistic execution, a complete trial log, and untouched out-of-sample evidence. realbacktesting describes its execution assumptions and validation layers on the methodology page. A cTrader-native report is reproducible evidence of what was tested; it is not a licence to forget how the strategy was selected.
Frequently asked
Is a high Sharpe ratio evidence of overfitting?
No. A high Sharpe ratio is evidence of strong risk-adjusted performance in the measured sample. Overfitting risk depends on the search process, sample uncertainty, return shape, and out-of-sample behaviour.
What counts as a trial in backtest selection?
A trial is a distinct candidate considered during research, including changes to rules, filters, markets, timeframes, or parameters. Correlated variants complicate the effective count, so the full search should be logged rather than reduced to the final file.
Does out-of-sample testing replace the deflated Sharpe ratio?
No. Out-of-sample testing evaluates a frozen strategy on untouched data. Deflated Sharpe evaluates whether the selected in-sample claim was inflated by the research process; the two checks answer different questions.
Can I calculate it if I did not save rejected tests?
Not credibly. An incomplete trial record understates the selection pressure and makes the result look more certain than the research process warrants.
The stubborn takeaway: the best Sharpe in the folder is not evidence until the folder itself is part of the test.