How can information about Backtesting Practice be verified?

Direct answer

Information about backtesting practice can be verified by checking (1) what exact method was used, (2) what inputs were assumed (data, costs, execution rules), (3) whether results can be reproduced from those inputs, and (4) whether known failure modes were tested. A claim is only meaningful if it is specific enough to re-run and independent enough that another person can reach the same conclusions under the same assumptions.

Definition and verification “what”

Backtesting practice is the process of evaluating a trading model or decision rule by applying it to historical data and simulating what would have happened under an explicit set of rules. Verification starts with separating:

• Stable mechanics: the algorithmic steps (entry/exit logic, position rules, risk calculations) and the calculation procedure.
• Variable conditions: market behavior over time, and any assumptions that can change the outcome such as spreads, commissions, slippage, liquidity, and execution timing.

A useful verification target is a complete specification. At minimum, the information should state which decision rule was tested, what data period(s) were used, how signals were generated in time, and how trades were simulated.

Evidence and example checks you can reproduce

Use a source hierarchy that matches how claims become testable:

1. Primary method description: the original description of the backtesting procedure (logic and formulas).
2. Input transparency: the historical data source and preprocessing steps (cleaning, resampling, timestamps).
3. Execution model: the rules for how orders would have filled (price source, latency assumptions, and how costs are applied).
4. Computation trace: enough detail to reproduce metrics from raw outputs (trade list, position sizing calculations, and aggregation method).

Reproducible verification steps:

• Step 1: Record assumptions. Write down every assumption that affects results: the time zone and candle alignment, whether the model uses close vs open, transaction costs, and whether fills assume the same-bar price is available.
• Step 2: Check event timing (look-forward vs look-back). Ensure that decisions at time t only use information that would have been known at time t. A common failure is using future bars to form features or to decide entries.
• Step 3: Validate the execution logic. Confirm how the model converts decision rules into trades: when positions change, how costs are deducted, and whether partial fills or slippage are ignored or approximated.
• Step 4: Recompute metrics from a trade log. If the backtest only reports summary statistics, ask for the underlying trade-by-trade outputs or recreate them from stated formulas.
• Step 5: Run a sensitivity test. Change one assumption at a time within reasonable ranges (for example, slightly different cost or fill assumptions) and confirm whether performance is robust or collapses.

Material limitation and failure modes to explicitly look for:

• Look-ahead bias (using information unavailable at decision time).
• Overfitting and repeated testing (tuning parameters until they match historical quirks).
• Unrealistic fills (assuming perfect execution without liquidity or slippage effects).
• Selection bias (choosing favorable periods after seeing results).

Because historical relationships do not guarantee future results, verification should focus on whether the method is internally consistent and whether it demonstrates resilience to plausible assumption changes.

Limitations and risks

Even when backtesting is performed carefully, results remain conditional on the stated assumptions. Costs, execution timing, and market microstructure can differ substantially across periods, and small modeling differences can produce large metric changes. Therefore, verification should not treat a strong backtest as evidence of future performance; it is evidence that a method can behave in a particular way under particular simulated conditions.

Verification checklist and next question

To verify a backtest claim independently, confirm that all of the following are stated clearly and can be re-run:

1. Data scope and preprocessing details.
2. Exact decision logic and how it maps to orders.
3. Execution timing assumptions and how prices for fills are selected.
4. Cost assumptions and how they are applied per trade.
5. Reproducible outputs (at least a trade log or calculation trace).

If any of these items are missing, you can still assess the claim qualitatively, but you cannot fully verify it. The next question to ask is: Can the same backtest be reproduced end-to-end using only the information provided, without adding hidden assumptions?