Mechanism: what “Strategy Review” means
A Strategy Review is a structured process to evaluate a previously defined trading approach using recorded inputs and measurable outcomes. Its purpose is not to predict future results, but to understand what happened, which assumptions mattered, and whether the recorded logic can be independently reproduced.
To assess it accurately, focus on data that describes four areas:
- the strategy definition and assumptions,
- the execution and cost details that affect realized results,
- the outcome measures that were computed,
- the provenance and quality of the data used to do the computations.
Data inputs you need
Collect the data that fully specifies both “what was planned” and “what actually occurred.” A practical set of inputs includes:
- Strategy specification (stable mechanics): the rules used to enter, exit, and manage positions; the timeframe; risk constraints; and any decision logic. Include assumptions (for example, how position sizing was calculated).
- Market and execution records: timestamps, instrument identifiers, order types, execution prices, and any partial fills. Execution matters because realized performance depends on how orders were actually filled.
- Costs and frictions: spreads (if relevant), commissions, financing/rollover charges, and other fees that reduce returns. If costs were modeled rather than observed, state the method.
- Position and equity accounting: trade ledger entries or an equivalent account statement, plus how the equity curve was constructed.
- Outcome calculations: the exact metrics used (for example, return measures, drawdown measures, win/loss statistics) and the formulas or spreadsheet logic.
For any calculation or example, state the assumptions explicitly, such as whether returns are net of costs, whether leverage affects exposure in a specific way, and what currency conversion was used if applicable.
Provenance and timeliness checks
A Strategy Review is only as reliable as the data pipeline behind it. Verify:
- Provenance: where each dataset came from (original trade blotter, platform export, manual logs, modeled backtest outputs). Record who produced it and how it was generated.
- Completeness: whether all trades and events in the review window are included, including re-quotes, cancellations, manual overrides, or missing days.
- Timeliness and alignment: confirm that timestamps are consistent across datasets (for example, time zones and bar alignment). If the review mixes bar data and execution data, document the mapping.
Even if the strategy rules are unchanged, results can differ due to data timing mismatches. Timeliness checks reduce the risk of “looking correct” while using misaligned inputs.
Quality checks to confirm the review is reproducible
Use quality checks that can be repeated without relying on the original author’s memory:
- Recompute metrics from raw inputs: verify that published performance figures match the underlying ledger and formulas.
- Unit and transformation validation: ensure prices, sizes, and currency conversions are consistently handled.
- Outlier and anomaly review: identify impossible values (for example, negative spreads if modeled incorrectly, missing execution prices, or equity jumps that do not match trade records).
- Method consistency: confirm that the same assumptions and costing method were used throughout the review period.
A useful test is to try an independent calculation of at least one key metric from the raw inputs, using the stated formulas and assumptions.
Limitations and material failure modes
Strategy Review data still cannot guarantee correct conclusions. Key limitations include:
- Non-stationarity: historical relationships may not persist under new market conditions.
- Cost and execution variability: spreads, commissions, and execution quality can change, so recorded performance may not reflect future realizations.
- Model risk: if parts of the data were simulated (modeled costs, assumed fills, or derived values), the uncertainty should be acknowledged.
- Survivorship and selection effects: if only profitable segments are included, the review becomes biased.
One material failure mode is “reproducibility failure”: the review cannot be independently verified because formulas, assumptions, or raw inputs are missing or unclear. Another is “misaligned data”: performance appears consistent until timestamps or mappings are corrected.
Verification and next questions
To verify a Strategy Review, ask what exact data produced each metric and whether you can recompute it from the documented inputs. If the review relies on modeled elements, ask what assumptions were used and how sensitive the results are to those assumptions.
Next, confirm that the review separates stable mechanics (rules and accounting logic) from variable conditions (market regime, costs, and execution conditions).