Backtesting practice in plain terms
Backtesting practice is the process of evaluating a trading idea (for example, a rule-based entry and exit) by running it on historical price data. The goal is to estimate what the idea might have produced in the past if the same rules were followed.
A backtest typically includes (1) historical data, (2) a rule or model that generates actions from that data, and (3) a way to calculate outcomes such as profit and loss, drawdown, and win/loss frequency. These calculations rely on assumptions about what happened inside each time period (for instance, how prices moved between recorded data points) and about trade execution (for instance, commissions, spreads, and whether orders were filled).
Why backtesting can be less useful than it seems
A key limitation is that the backtest is a simulation of the past, not an observation of the future. Even when historical outcomes look strong, relationships learned from history may not repeat under new market conditions. The limitations below explain common failure modes.
1) Historical data and market microstructure gaps
Most backtests use recorded price bars or ticks, but real trading involves more detail than what many datasets capture. If your historical data is incomplete, adjusted in a way that changes the meaning of price, or lacks information needed to model order fills, the backtest may deviate from what would happen in live trading.
Also, the mapping from recorded prices to possible fills is uncertain. For example, if a rule assumes a fill at a specific price level, but the real market could have moved past that level before an order could be executed, the simulation becomes optimistic.
2) Costs, execution quality, and slippage assumptions
Even if the strategy logic is correct, results depend heavily on assumptions about transaction costs and execution. Backtests sometimes use simplified cost models (for example, constant spreads or no slippage) or assume perfect order filling. In practice, spreads widen, liquidity changes, and slippage can vary with volatility and time of day.
Because small cost differences can compound over many trades, a backtest that ignores or underestimates these effects can overstate performance.
3) Overfitting and data-snooping
Backtesting can unintentionally “fit” the noise of a particular dataset. This happens when many parameters are tried and selected because they improved past results. The more you search for a combination that looks good on the same data, the greater the chance that the apparent edge is specific to that period rather than a general property of the market.
A related issue is data-snooping: repeatedly testing many variants until one appears to perform well. Without careful separation of training and testing periods, strong past results may not reflect a repeatable mechanism.
4) Non-stationary markets and changing conditions
Markets are not stable systems. Volatility regimes change, participation shifts, and liquidity can move. A rule that worked during one environment may behave differently when spreads, volatility, or correlations change.
Even without regulatory or structural events, the statistical properties of price series can drift. That means the assumptions that were implicitly “baked in” during the period you tested may no longer hold.
5) Ambiguous definition of what exactly is being tested
Backtests can be difficult to compare because the inputs and rules differ. For any example, you should be able to state:
- What data frequency was used (for example, bars vs. ticks).
- How indicators or features were computed and whether they used only information available at the time.
- How entries and exits were decided and how order fills were modeled.
If any of these details are unclear, the backtest may not test the intended idea; it may instead test an artifact of the chosen implementation.
Verification that can improve reliability
You cannot make a backtest “prove” that a strategy will work, but you can check whether results are likely to be robust.
A practical verification approach is to define assumptions explicitly and test sensitivity. For example, vary cost parameters within reasonable ranges, or use different historical periods with different market conditions. Separately, use out-of-sample testing: evaluate the rules on data not used for parameter selection.