The Probability of Backtest Overfitting
Introduced combinatorially symmetric cross-validation (CSCV), a way to estimate the chance that a backtested strategy is fitted to its own history. The insight behind it is simple. Optimizing over the same dataset again and again inflates how good the result looks. CSCV measures that inflation by checking where in-sample winners land across all C(16, 8) = 12,870 possible in/out-of-sample splits.
The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality
Introduced the Probabilistic Sharpe Ratio (PSR) and the Deflated Sharpe Ratio (DSR). PSR gives the probability that an observed Sharpe ratio really exceeds a benchmark, after correcting for lopsided and fat-tailed returns (skewness and excess kurtosis) and for the number of strategy trials tested. DSR then marks the observed Sharpe down by a correction factor derived from the Euler-Mascheroni constant, which accounts for picking the best of many backtests.
The Sharpe Ratio Efficient Frontier
Derived the Minimum Backtest Length (MinBTL). That is how many observations you need before a given Sharpe ratio means anything, given the number of strategy configurations tested and the shape of the return distribution. This formula is the direct source of OverfitCheck's Sample Size / Minimum Backtest Length test threshold.
... and the Cross-Section of Expected Returns
Examined 316 published cross-sectional return predictors in the academic literature. Found that the usual t-statistic bar of 2.0 is too low once you count how many things the profession collectively tested. The adjusted bar should be roughly 3.0. That 3-sigma standard sets the significance thresholds in OverfitCheck's Deflated Sharpe and Sample Size tests.
Backtesting
Proposed a framework for judging backtested trading rules when many rules have been tested. Argued that many strategies published in academic and practitioner literature would not survive that correction, and that practitioners routinely undercount the trials they have actually run.
Advances in Financial Machine Learning
Book-length treatment of machine learning in quantitative finance. Chapter 12 sets out the walk-forward testing method, including the in-sample/out-of-sample split and the degradation ratio calculation, that OverfitCheck implements in its Walk-Forward Degradation test.
Implementation note
OverfitCheck does not use mlfinlab or any proprietary library. All 8 tests are written directly from the formulas in the papers above and checked against the numerical examples those papers publish. The whole engine was then calibrated against six real strategy trade logs of known ground truth, where it reproduced the true quality ordering exactly (see the calibration writeup). Bootstrap 95% confidence intervals are computed natively using NumPy and SciPy.