How many did you try?
Ask any backtest one question.
Someone shows you a brilliant track record. Type in how many strategies were tried and how many years it covers. See if luck alone would have got there.
Return divided by bumpiness, per year. Above 1 is what funds put in bold.
Count the ones nobody wrote down. Nobody knows? Try a few guesses.
Why the count matters
Try one strategy and a high Sharpe ratio is rare by luck. Try a thousand and the best of them is high every time, even if none of them can make money. The grey hill above is where that best one lands when nothing works. A claim inside the hill could be pure luck.
The count is the number that never makes the brochure. Every rule tested, every setting tweaked, every idea dropped quietly counts. If many tries were close copies of each other, count them as fewer.
This is how the professionals check
Bailey and López de Prado built the "deflated Sharpe ratio" on exactly this idea: judge the best result against the best that luck would give after the same number of tries. Harvey, Liu and Zhu looked at 316 published stock-market factors and argued a new one needs a test score above 3, not 2, because so many people had been searching. Hou, Xue and Zhang re-tested 447 published patterns and 85 percent failed that stricter bar.
A test score is the Sharpe ratio times the square root of the years. So a score of 3 over 9 years needs a Sharpe ratio of 1.
Before you trust a track record, ask: how many did you try? Then put the answer in here.
Sources
- Bailey and López de Prado, "The Deflated Sharpe Ratio", Journal of Portfolio Management (2014). The expected best Sharpe ratio from many tries, used for luck's usual best.
- Harvey, Liu and Zhu, "…and the Cross-Section of Expected Returns", Review of Financial Studies (2016). 316 factors; new factors need t above 3.0.
- Hou, Xue and Zhang, "Replicating Anomalies", NBER working paper 23394 (2017). 447 anomalies; 380 (85 percent) fail with a t-cutoff of 3.
- The chance on this page treats each try as an independent strategy with no edge, using a normal approximation to the Sharpe ratio.