In short
To backtest a trading rule without fooling yourself, treat a good result as a question, not an answer. Most backtests look great because the rule was shaped to fit past prices. Split your data so you build on one part and test on another you never saw. Keep the sample large, because a handful of trades is luck, not proof. Subtract real fees, slippage and funding, since they quietly eat paper profits. Then forward test on paper before any money is at risk. An honest backtest tells you what could fail, not what will work.
What does a backtest actually prove?
A backtest proves only that a rule would have worked on data you already have. It cannot prove the rule works in the future, because markets change and the past is a sample, not a law. Treat a strong result as a hypothesis to stress, never a promise to trust.
A backtest replays a rule over historical prices and reports how it would have done. That is useful, because it filters out ideas that never worked even on paper. It is also dangerous, because the same data that trained the rule is being used to grade it. A result built and judged on one dataset flatters itself.
What is different here
The ParadiseTeam treats every promising rule as a hypothesis to break, not a signal to trust. We stress it against out of sample data and years of market history before it shapes a single real position.
The seven ways a backtest lies to you
Most backtesting failures are not bad math. They are honest mistakes that make a weak rule look strong. Learn the seven common traps and you can design a test that fights back against your own optimism.
| The trap | How it fools you |
|---|---|
| Curve fitting | Extra rules fit past noise, not future signal. |
| Cherry-picked dates | A kind date range hides the ugly ones. |
| Tiny sample | Too few trades turn luck into false proof. |
| Ignored costs | Fees, slippage and funding erase paper gains. |
| Survivorship bias | Testing only coins that survived skips the dead. |
| Look-ahead bias | Using data you could not have known yet. |
| No out-of-sample check | Grading the rule on the data that built it. |
Why does curve fitting make the future worse?
Curve fitting means adding rules and parameters until the backtest fits past prices almost perfectly. Each extra condition carves the result closer to history, including its random noise. The future does not repeat that exact noise, so the over tuned rule breaks live. A simple rule that works loosely usually beats a complex one that fits perfectly.
Statisticians call this overfitting, and it is the single most common way a trader fools themselves. If your rule needs five filters and three exact thresholds to look good, it is probably memorising the past. A model that fits the past perfectly has, in a sense, learned nothing useful at all. Cut parameters until the edge is boring and robust, not dazzling and fragile.
How do you split data so you cannot cheat?
Split your history into two parts before you build anything. Use the first part, the in sample data, to design and tune the rule. Keep the second part, the out of sample data, untouched until the rule is final. Then run it once on that unseen data. If the edge survives, it is more likely real than remembered.
The discipline matters because your brain will cheat without meaning to. Once you have seen the whole dataset, every tweak is secretly informed by the answer. A clean out of sample test is the closest a solo trader gets to an honest referee. Run it only once, because repeated peeking slowly turns test data into training data.
Why is a handful of trades not evidence?
A handful of trades cannot tell skill from luck. With ten trades, a 60 percent win rate could easily be random, and often is. You need a large enough sample that the result would be unlikely by chance alone. As a rough floor, look for dozens of trades at minimum, and treat hundreds as far more trustworthy.
This is also why patience protects your test. A rule that only fires a few times a year needs many years of data to prove anything. Forcing more trades to fill the sample just adds noise, which is why sitting out is a position. Fewer, cleaner trades beat a padded log every time.
How do fees and slippage erase paper gains?
Every real trade pays fees, loses a little to slippage, and on leverage pays funding. A backtest that ignores these reports profits that never existed. Many rules that look green before costs turn red after them, especially high frequency ones. Always subtract realistic fees and slippage per trade, then judge what is left, not the gross number.
Slippage is the gap between the price you expected and the price you actually got. It grows when markets move fast or liquidity is thin, exactly when many signals fire. Model it honestly, and size positions so costs stay small against your edge. This connects directly to how you size risk per trade, because leverage multiplies both the edge and the drag.
Survivorship and look-ahead bias: testing on a map of the future
Survivorship bias is testing only the coins that are still around today. The tokens that went to zero quietly vanish from your dataset, so your rule looks safer than reality. If you backtest a buy the dip rule on today’s top coins, you are studying the winners of a game already decided. Include the dead names, or accept your result is survivorship bias in disguise.
Look-ahead bias is sneaking in data you could not have known at the time. Using a daily closing price to decide a trade at midday is a classic example. So is using a revised figure that was only published later. Every decision in your test must use only what was actually visible at that exact moment.
Building an honest test: a step by step checklist
An honest test is mostly about removing ways to cheat before you start. Work through this checklist in order, and do not skip the boring steps.
- Write the rule down fully before you touch any data.
- Split history into in sample and out of sample sets.
- Build and tune only on the in sample data.
- Subtract real fees, slippage and funding on every trade.
- Check the sample is large enough to rule out luck.
- Run the final rule once on the untouched data.
Treat the result as a probability read, not a forecast. A rule that passes is a candidate worth trusting a little, not a machine that prints money. The same honesty applies when you read a probability model: it describes odds, never certainty.
Why forward test on paper before risking capital?
Forward testing means running the finished rule on new, live prices without real money. It catches problems a backtest cannot, like slow fills, spread changes and your own hesitation. Run it for weeks or months, long enough to collect a fair sample of live trades. Only after it holds up should any real capital follow, and then in small size.
Paper trading also tests you, not just the rule. Many strategies that backtest well fall apart because the trader cannot follow them under stress. Watch how you behave when a fakeout traps a breakout, or when you must decide on taking partial profits. If you cannot run the rule calmly on paper, live money will not fix that.
Frequently asked questions
How many trades do I need for a reliable backtest?
There is no single magic number, but a few dozen trades is a bare minimum, and hundreds is far safer. The rarer your signal, the more years of data you need. With too few trades, a good win rate is probably luck, not a real edge.
What is the difference between in sample and out of sample data?
In sample data is the history you use to build and tune a rule. Out of sample data is a separate slice you keep hidden until the rule is final. Testing on the out of sample set shows whether the edge is real or just fitted to the past.
Why does my backtest look better than my live results?
Usually because the backtest ignored real costs, used tuned parameters, or peeked at data it could not have known live. Fees, slippage and curve fitting all flatter paper results. The gap between test and reality is the exact self deception an honest backtest is designed to expose.
Is backtesting worth it for crypto trading rules?
Yes, as a filter, not a crystal ball. A backtest quickly kills ideas that never worked, even on friendly data. It cannot promise future results, because crypto markets shift and the past is only a sample. Use it to reject weak rules, then forward test the survivors on paper.
Crypto trading involves substantial risk and is not suitable for everyone. Nothing here is financial advice; it is education only. Never risk more than you can afford to lose.
Join the discussion
No comments yet. Members, share how you are reading this.