Documentation/Research & papers
Research

Research & papers

Every OverfitCheck test comes straight out of published, peer-reviewed work. Every method, formula, and threshold traces back to one of the six sources below. The preprints are free to read.

PBO test

The Probability of Backtest Overfitting

Bailey, D., Borwein, J., López de Prado, M., & Zhu, Q.
Journal of Computational Finance · 2015

Introduced combinatorially symmetric cross-validation (CSCV), a way to estimate the chance that a backtested strategy is fitted to its own history. The insight behind it is simple. Optimizing over the same dataset again and again inflates how good the result looks. CSCV measures that inflation by checking where in-sample winners land across all C(16, 8) = 12,870 possible in/out-of-sample splits.

Deflated Sharpe test

The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality

Bailey, D., & López de Prado, M.
Journal of Portfolio Management · 2014

Introduced the Probabilistic Sharpe Ratio (PSR) and the Deflated Sharpe Ratio (DSR). PSR gives the probability that an observed Sharpe ratio really exceeds a benchmark, after correcting for lopsided and fat-tailed returns (skewness and excess kurtosis) and for the number of strategy trials tested. DSR then marks the observed Sharpe down by a correction factor derived from the Euler-Mascheroni constant, which accounts for picking the best of many backtests.

Sample Size test

The Sharpe Ratio Efficient Frontier

Bailey, D., & López de Prado, M.
Journal of Risk · 2012

Derived the Minimum Backtest Length (MinBTL). That is how many observations you need before a given Sharpe ratio means anything, given the number of strategy configurations tested and the shape of the return distribution. This formula is the direct source of OverfitCheck's Sample Size / Minimum Backtest Length test threshold.

Deflated Sharpe testSample Size test

... and the Cross-Section of Expected Returns

Harvey, C. R., Liu, Y., & Zhu, H.
Review of Financial Studies · 2016

Examined 316 published cross-sectional return predictors in the academic literature. Found that the usual t-statistic bar of 2.0 is too low once you count how many things the profession collectively tested. The adjusted bar should be roughly 3.0. That 3-sigma standard sets the significance thresholds in OverfitCheck's Deflated Sharpe and Sample Size tests.

Multiple testing context

Backtesting

Harvey, C. R., & Liu, Y.
Journal of Portfolio Management · 2015

Proposed a framework for judging backtested trading rules when many rules have been tested. Argued that many strategies published in academic and practitioner literature would not survive that correction, and that practitioners routinely undercount the trials they have actually run.

Walk-Forward test

Advances in Financial Machine Learning

López de Prado, M.
Wiley · 2018

Book-length treatment of machine learning in quantitative finance. Chapter 12 sets out the walk-forward testing method, including the in-sample/out-of-sample split and the degradation ratio calculation, that OverfitCheck implements in its Walk-Forward Degradation test.

Implementation note

OverfitCheck does not use mlfinlab or any proprietary library. All 8 tests are written directly from the formulas in the papers above and checked against the numerical examples those papers publish. The whole engine was then calibrated against six real strategy trade logs of known ground truth, where it reproduced the true quality ordering exactly (see the calibration writeup). Bootstrap 95% confidence intervals are computed natively using NumPy and SciPy.