Essays · The method

Why most backtested trading edges are statistical mirages

By Daniel Roache · 14 July 2026 · 8 min read

You've seen the screenshot. A smooth equity curve climbing left to right, a caption that reads "97% win rate," and a link to a Discord. The chart is real. The edge is not. The gap between those two facts is where most retail capital goes to die.

This essay is about that gap — specifically, about why a backtest that looks fantastic tells you almost nothing, and what has to be true before a result is worth a single dollar of real risk.

The base rate: assume it's noise

Start every test from the same place: this edge is noise until proven otherwise. That isn't pessimism, it's arithmetic. Financial markets are close to random over short horizons, and historical price data is finite. If you try five hundred variations of a rule, one of them will look brilliant on that specific slice of history — not because it captured a market truth, but because it captured that slice's noise. This is the core of backtest overfitting: the more you optimise, the better the backtest looks and the worse the strategy actually performs.

The job of a backtest isn't to confirm your idea. It's to give the idea every honest chance to fail. If it survives that, you have something. If it doesn't, you just saved yourself the tuition.

How a mirage is manufactured

Mirages don't usually come from fraud. They come from ordinary, well-intentioned mistakes that each nudge the result upward:

• Look-ahead fills. The moment you let a trade fill on the same bar that generated the signal, you've booked profit from information you didn't have yet. This single bug flatters more strategies than any other. Honest tests fill at the next bar's open.
• Ignoring fees. A round-trip costs real money — on crypto perps, roughly 0.2R per trade once you include the spread. Any gross "edge" smaller than the fee wall is structurally negative; it just doesn't know it yet. A shocking number of published systems live below this line.
• Counting correlated trades as independent. Ten "wins" that were really one correlated event should count as one bet, not ten. Autocorrelation inflates the trade count, which inflates the apparent statistical significance.
• Over-search. Grid-searching thousands of parameter sets and reporting the best one is the textbook recipe for a curve fit. If your rule contains an oddly specific number — enter when RSI crosses 67.3, not 70 — ask why that number, and whether it survives being 65 or 71.

A backtest that assumes perfect fills at the price you wanted is fiction. The first job of an honest harness is to make fills expensive and pessimistic.

The tells of a too-good-to-be-true result

Before you trust any equity curve, run it past this list. Professional quant desks target Sharpe ratios around 1–2. If a retail backtest shows 5-plus, something is wrong, not miraculous.

• An equity curve far smoother than the noise in its own individual trades.
• A big gap between in-sample and out-of-sample performance.
• Results that collapse when a parameter moves one notch — a peak, not a plateau.
• Profit that disappears when you remove the top three trades.

Each of these looks like a winning strategy on a naive chart. Each dies under scrutiny. We walk through all ten of them in 10 signs your trading strategy is overfit.

What actually separates an edge from a mirage

One question does most of the work: which market conditions did this strategy never see, and did it still work? A trend-follower looks like a genius in a bull market and gives it all back in a range. If you fit and test in the same regime, you'll never notice. The fix is to hold out whole market regimes the strategy was never trained on and score it there — the discipline we call Leave-One-Regime-Out, part of the two-tier harness every idea here runs through.

An edge that clears next-bar fills, one position at a time, a real t-statistic on independent trades, netted fees, and profitability across held-out regimes it never saw — that's rare. It's also the only kind worth funding. Everything else is a screenshot.

Keep reading