Every test is built from the original paper, not a simplified stand-in. The math runs on your full trade log, not on the summary numbers your platform prints at the top.
train/test splits behind the overfitting test on a 16-configuration upload. Each split asks one question: did your in-sample winner survive?
reshuffles of your trade order in the Monte Carlo test. If shuffled versions beat you often, luck explains your returns.
real strategies of known quality, graded by the engine in the exact order their real-world verdicts demanded.
slowest full audit in that calibration run. The fastest took 12.6 seconds, on real backtests of 164 to 1,111 trades.
Every audit starts at 100 points. Each test takes points off when it finds something, and the amount is fixed in advance. Nothing is weighted by how the result looks.
That is on purpose. A backtest can lose more points than it has, and the score stops at 0 rather than going negative. It is also why two very different failures can both read 0 / F. Transaction costs are the one test with no penalty attached. Broker fills vary too much for a fair universal deduction, so we show you the number and leave the decision where it belongs. See the exact thresholds for every test →
Test 50 parameter combinations and keep the best one, and you have not found an edge. You have found the combination that got lucky. The test for this is called PBO. It splits your backtest into 12,870 different pairings of a training half and a test half.
Each split asks the same question. Did the combination that won in training also win in testing? The share of splits where it did not is your PBO. Above 50%, your best strategy was more likely picked by luck than by skill.
Catches: parameter optimization overfitting, multiple comparisons bias, in-sample bias.
A real edge gives back a little on new data. It does not collapse. So we hold back the last 30% of your backtest. Your strategy never touched that stretch during development. Then we measure how far performance falls when it gets there.
A degradation ratio above 0.8 means less than a fifth of the performance went missing. That is acceptable. Below 0.2 is critical and takes the maximum penalty. Most retail backtests land under 0.2. In plain terms, 80% of the returns you were counting on were never there.
Catches: in-sample overfitting, regime-specific fitting, forward-looking data leakage.
Your backtest made money. The open question is whether that took skill or a good run. We shuffle the order of your actual trades 10,000 times and count how often a random ordering beats you.
If 31% of random orderings beat you, your p-value is 0.31. Not significant. Below 0.05 passes cleanly. Above 0.10 takes the full penalty. The test ignores the shape of your equity curve on purpose. It asks one thing. Do the trades you picked show skill?
Catches: equity curve shape overfitting, lucky trade sequences, path-dependent strategies.
Your raw Sharpe ratio is almost certainly too high. Bailey and López de Prado (2014) showed that every extra configuration you try raises the best Sharpe you would expect from pure chance. A Sharpe of 1.4 after 30 tries can be nothing but noise.
The Deflated Sharpe Ratio corrects for that. It works out the lowest Sharpe that would still mean something, given how many configurations you tested. Clearing the raw bar is easy. Clearing the deflated bar is the part that counts.
Catches: multiple testing bias, inflated performance from extensive optimization.
Markets move between three moods. Trending, when prices run in one direction. Ranging, when they chop around a mean. High-volatility, when uncertainty spikes and news drives everything. A strategy that prints money in one of them can bleed in another.
We label each quarter of your backtest by regime, then compute returns inside each one separately. A strategy that only works in trending markets is not a diversified edge. It is a bet that one market condition keeps going.
Catches: regime-specific overfitting, strategies that look diversified but concentrate regime exposure.
Statistical claims need enough trades behind them. Bailey and López de Prado (2014) derived the minimum number of trades a Sharpe ratio needs before it means anything, given how many configurations you tested.
Test 50 configurations and keep the best, and you need far more trades than someone who ran one strategy once. 30 trades is not enough for most real situations. We show you where your strategy sits on that curve, and how far short it falls if it does.
Catches: over-optimistic significance claims from small sample sizes, underpowered tests.
Most backtests assume perfect fills at the bid or ask with zero commission. Real trading pays slippage on the way in and the way out, plus commission and spread. Small per trade. Over hundreds of trades, a serious drag.
We rebuild your equity curve with realistic cost estimates for your asset class: futures (0.05–0.1% slippage plus exchange fees), forex (1–2 pip spread), crypto (0.05–0.2% exchange fee plus 0.1–0.5% slippage). The two curves, before and after, show what execution actually takes.
Catches: strategies that only work with zero-cost assumptions, high-frequency strategies with small per-trade profit.
A real edge does not care whether your RSI period is 14 or 13. A strategy that works at 14 and falls apart at 12 or 16 is not a discovery. It is a curve fitted to history. We move each parameter by ±20% and watch what the Sharpe does.
A narrow peak with steep sides is a warning. A flat plateau, where the strategy performs about the same right across the range, is evidence the edge does not live or die on one magic number.
Catches: curve-fitted parameters, strategies dependent on exact magic numbers.
BacktestBase is the closest thing to a competitor. “Manual Python” means building these tests yourself from the papers, which is a real option and the reason we rate the difficulty honestly instead of pretending it cannot be done.
One claim no competitor makes. We ran the engine on six real strategies whose quality we already knew: a live, validated edge and the strategies its research program killed. The killed ones graded F. The live edge graded B. Sorting all six by the engine's continuous outputs reproduced the real-world ordering exactly. The same report lists what the engine gets wrong. Read the calibration report →
Three ways in: a TradingView XLSX export, a MetaTrader 5 Strategy Tester report, or a CSV trade log from anything else. All three run the same eight tests on the same engine.
XLSX from the Strategy Tester. Upload it exactly as exported.
The Strategy Tester HTML report. Every trade we parse is checked back against the report’s own balance column.
One row per trade with a P&L column. We fuzzy-match the column names.
Export a CSV and it works today. Direct API import lands in V1.5.
Export a CSV and it works today. The native format lands in V2.
Send the file as your platform exported it. Opening and resaving in Excel or Sheets can break the formatting we read. The upload limit is 20 MB. Ten completed trades is the hard floor, and 30 to 50 is where the other tests start to mean anything. Below that we still run, and we say plainly that the evidence is too thin. Step-by-step export instructions →
Four capabilities specced from findings of the research program behind our calibration set. They are listed so you can see where this is going. None of them is for sale, and none will be until it exists and has been checked on real logs.
Timestamp forensics on your trade log. Does the backtest trade the session you think it trades? This catches daylight-saving drift and server-clock changes. That class of defect quietly shifts years of backtest trades by an hour, and nothing in your platform tells you.
Costs measured against each trade’s own stop distance instead of as a haircut on returns. You get a breakeven win rate per era to hold against your realized win rate, and a flag on any era that was unpayable before the first trade was placed.
Finds fixed parameters whose meaning drifted over a long backtest. A stop set in price units, on an instrument that grew several-fold, makes the early and late halves of your test two different strategies wearing one name.
Thousands of resampled challenge and funded-phase paths against versioned prop-firm rulesets, floating drawdown included. Payout accounting is strict: only swept payouts count, and every payout shrinks your buffer. You get P(pass), P(breach), and the risk per trade that actually maximizes expected cleared payout.