Why your backtest lied to you

Every algorithmic trader has the same photograph in their history: a backtest equity curve rising at forty-five degrees, drawdown under five percent, profit factor above two. Then live for six weeks, and the curve is flat or worse. The strategy did not stop working. It never worked — the test was measuring something other than reality.

1. Spread modelling

MT4's default tester uses the current spread, or a fixed value you supply. Real spreads breathe: two pips one moment, twelve at the New York close, thirty during a payrolls release, and wider still in the thin hour before Tokyo opens.

If your average target is fifteen pips, spread is not a rounding error — it is a material share of every trade. A system tested at a fixed two-pip spread and traded through an average of four is losing two pips per trade it never accounted for. Over four hundred trades, that alone can erase the edge.

How to check

• Use real tick data with variable spread, not the broker's default bars.
• Re-run the test at 1.5x and 2x your assumed spread. If the edge disappears, it was never robust.
• Log real spread on your own account for two weeks, at the hours you actually trade. Use that number, not the marketing one.

2. Slippage and execution assumptions

The tester fills you at the price you asked for. Reality does not. Stops especially: once triggered, a stop becomes a market order, and in a fast tape the next available price can be well away from the level.

The systems most damaged by this are the ones that look best on paper — high frequency, small targets, tight stops. The tighter your stop relative to the instrument's noise, the more of your result is decided by execution rather than by your idea.

Add a pessimistic slippage assumption deliberately. If the system only survives with zero slippage, you have not found an edge — you have found a rounding artefact.

3. Overfitting, the quiet one

This is the hardest to see because it does not feel like cheating. You test a moving average period, 20 is mediocre, 22 is better, 23 is excellent. You keep 23. Repeat across six parameters and you have not built a strategy — you have described the past in unusual detail.

The parameter sensitivity test

A robust parameter sits on a plateau, not a spike. Vary each input by ±20% and look at the result surface:

• Plateau — neighbouring values all perform reasonably. The edge is real and tolerant.
• Spike — only one value works, neighbours collapse. That is curve fitting, and it will not repeat.

Anyone can produce a spike. Nobody can trade one.

If your system needs the exact parameter you found, you did not find it — you built it out of noise.

4. Look-ahead bias

Subtler, and usually a coding bug rather than a judgement error. Using the current bar's close in a decision made during that bar. Referencing an indicator value that only completes at bar close while trading intrabar. Using next-day data in an aggregate.

The tell is a result that is too good — win rates above 80% with meaningful reward-to-risk, or drawdowns that look implausibly shallow. When a result seems extraordinary, suspect the code before celebrating.

5. Reporting only the survivor

You test forty variants, one is excellent, you keep it and discard the rest. But if you generate forty random variants, some will look excellent by chance alone. Unless you record how many you tried, you cannot tell luck from edge.

Keep a log of every variant tested. The count itself is data.

A test protocol that survives contact

1. Split the data. Build on the first 60%. Do not look at the rest.
2. Validate out-of-sample. Run once on the held-back 40%. Once. If you tune after seeing it, it is no longer out-of-sample.
3. Walk forward. Optimise on a rolling window, trade the next window, roll on. This is the closest offline approximation of real use.
4. Stress the assumptions. Double the spread, add slippage, delay entries by one bar.
5. Forward test on demo. Two months minimum, on the broker you will actually use.
6. Go live small. Smallest size that is still real money. Live results are the only ones that count.

Most systems die at step two, and that is the point. A test protocol is not there to confirm your idea — it is there to kill weak ideas cheaply, before the market does it expensively.

Once a system is worth coding properly, structure matters as much as logic — see building your first EA: structure before strategy. And for why execution assumptions differ so much between brokers, read broker, LP, prime broker.