Free course·5 lessons

How to tell if your backtest is real

Five short lessons. No sign-up, no email, nothing to download.

Almost everyone who builds a trading strategy arrives at the same moment. The backtest looks good. Sometimes it looks extraordinary. The question is whether that result means anything, and the honest answer is that a good backtest is weak evidence on its own.

This course is about telling the difference. It is not about finding strategies and it will not teach you entries or indicators. It covers the specific ways a backtest ends up describing the past instead of predicting the future, and what you can check yourself before any money is involved.

Before you start, the honest part

Most people who trade retail accounts lose money. This is not a motivational setup where the next sentence explains why you will be different. It is the base rate, it is consistent across brokers and countries and decades, and nothing on this site changes it.

The same is true of prop-firm evaluations. The great majority of people who buy a challenge never receive a payout. Some fail on the rules rather than the trading, which is worth understanding before you pay a fee, but most simply do not have an edge large enough to survive the constraints.

Building and backtesting a strategy is the easy part of this. It is also the enjoyable part, which is why so many people spend years there. The difficulty is in the gap between a backtest and a live account, and most of this course is about that gap.

The point of validating anything

A test that tells you not to trade a strategy has done something useful. It has saved you the money you were about to lose finding out the slow way. If you take one idea from this course, that is the one: deciding not to trade is a result, not a failure.

It is also reasonable to decide this is not for you. If the honest version of the process sounds tedious rather than interesting, that is useful information. The work is mostly checking, waiting and discarding, and the people who do well at it tend to be the ones who find that part tolerable.

The lessons

01What overfitting actually is
You tried a lot of settings and kept the best one. The winner’s results include the luck of having won, and that luck does not come with you.
02Costs are the first filter
Spread, commission and overnight financing kill more candidate strategies than statistics ever will. You can check yours with arithmetic today.
03How many trades you actually need
Thirty trades tells you close to nothing. The number you need depends on how many variations you tried, and the answer is larger than most people expect.
04Normal looks like failure
Long flat stretches happen to strategies that are working. If you have not decided in advance what a real breakdown looks like, you will change a working system during an ordinary bad patch.
05What to do before you risk money
The checklist version. Write the rules down first, hold data back, count your variations, and size so that being wrong is survivable.

Read them in order if the subject is new. Each one also stands on its own, and each defines its terms as it goes.

Where the examples come from

The worked examples in this course are illustrations with round numbers, chosen so you can redo the arithmetic on paper. Where a lesson refers to something that happened in real research, it links to the write-up on the blog, which covers a data bug that went unnoticed for a decade, a set of losing years that turned out to be a cost problem, and a battery of eleven popular strategy improvements of which ten made things worse.

Nothing here requires an account. If you want the method behind the statistical tests themselves, that is in the documentation.