What are the rules of Strategy Review?

Direct answer

Strategy Review is a structured process for evaluating a trading approach by checking whether its stated rules and assumptions match observed outcomes. The “rules” of Strategy Review are not a guaranteed method for profit. Instead, they are constraints that make the evaluation testable: you define terms up front, you separate what is part of the approach (stable mechanics) from what comes from the market or execution (variable conditions), and you document assumptions so the analysis can be repeated. It also includes a deliberate search for failure modes—ways the review could be wrong or incomplete.

Mechanism or definition: the rule set

A testable Strategy Review usually follows a rule set that can be written down and applied consistently.

Define the strategy elements before evaluating results Write down what the approach actually is in rule form: the decision logic (entry/exit criteria), the position sizing logic (if any), and the management logic (how it behaves after entry). If you cannot describe these mechanics precisely, you cannot reliably test them.
Separate stable mechanics from variable conditions Treat market conditions and provider/execution details as variable inputs, not as part of the strategy. In practice, this means you distinguish:

Stable mechanics: what the strategy does according to its rules.
Variable conditions: bid/ask spread, commission, slippage, liquidity, rollover effects, and any data-quality differences.

This separation matters because historical performance can change when costs or execution change, even if the decision rules did not.

Set assumptions explicitly for every calculation or example For any performance calculation you use during the review, record the assumptions: the time period, how trades were timestamped, the cost model (fees and typical spreads, if included), and any data-cleaning steps. If you use hypothetical fills, state that they are assumptions and not real-time results.
Use consistent data selection rules Define which trades qualify for the evaluation and why. For example, specify whether you include only fully closed trades, how you handle partial exits, what you do with missing data, and whether you exclude periods where the strategy was not actually following its rules.
Test the strategy against evidence that can falsify it A Strategy Review is stronger when it asks, “What evidence would make this conclusion wrong?” For instance, if your review concludes that a certain behavior improves outcomes, you should test whether the improvement disappears under different but clearly defined conditions.

Evidence or example: a concrete, testable template

Below is an example of a rule-like Strategy Review workflow that someone else could replicate. It does not claim profit; it focuses on structure.

Assumptions for the example (you must document these in your own review):

No real-time market data is assumed.
Outcomes depend on market conditions, costs, execution quality, and jurisdiction.
Historical relationships do not guarantee future results.

Step A: Write the strategy mechanics Create a one-page “strategy definition” that includes:

What triggers decisions (the exact rule conditions).
What happens next (order type, timing conventions, exit rules).
Any constraints (risk limits, maximum holding time, or “no trade” conditions).

Step B: Record the evaluation inputs Define the evaluation inputs:

The dataset window (start/end dates).
The trade list construction rules (which trades are included).
The cost assumptions (commissions, spreads, or an explicit “costs excluded” choice).

Step C: Compute outcome metrics using the same assumptions Choose a small set of metrics relevant to testing the mechanics (for example, distribution of returns, drawdown behavior, or consistency of outcomes). The key rule is not which metric you pick, but that:

You compute metrics using the documented assumptions.
You do not mix metrics calculated with different cost models.

Step D: Compare mechanics-to-outcomes relationships under controlled changes To avoid accidental conclusions, vary one factor at a time while keeping the strategy rules constant. Examples of factors you might vary in analysis (without claiming predictive power):

Change only the cost assumption model and observe whether outcomes are sensitive.
Change only the evaluation window length and observe whether results stabilize.
Change only the execution timing assumption (e.g., conservative fill vs idealized fill) to see how robust the conclusions are.

Step E: Identify material limitations and failure modes A complete review includes at least one limitation that can explain the observed results without confirming the strategy’s intended logic.

Limitations and risks: where Strategy Review can fail

At least one material failure mode should be considered in every Strategy Review.

Data and execution mismatch If the dataset does not represent realistic fills, the review can overstate performance. Even when the strategy mechanics are correct, execution differences (slippage, spread changes, order timing) can change results.
Hidden rule changes Many strategies “drift” over time. If the rules used during the backtest differ from the rules you believe you are evaluating, the review tests the wrong thing.
Overfitting to history A review can become a fitting exercise: adjusting parameters until past results look good. This reduces testability and makes conclusions less likely to generalize.
Cost model inconsistency If costs are omitted or modeled differently across tests, comparisons are unreliable. A strategy may appear effective only because costs were not included consistently.
Survivorship and selection bias If you only include trades or periods that look favorable, or exclude problematic data without documented rules, the review becomes non-independent.
Jurisdiction and regulatory context Trading activity occurs within specific legal and regulatory constraints that can differ by region. Strategy Review cannot be divorced from those constraints if they affect how trades can be placed, reported, or executed.

Verification or next question: how to independently check

To verify the relevant facts in a Strategy Review, someone else should be able to repeat the logic from your documentation without access to your private reasoning.

Use these verification rules:

Recreate the same strategy definition and the same dataset selection rules.
Use the same documented assumptions for costs and execution modeling.
Check that computed metrics align with those assumptions.
Identify at least one falsification test: a specific change to inputs or conditions that would weaken your conclusion.

A next question to consider is: “Which parts of my review conclusion depend most on assumptions?” If the answer is “almost everything,” the review is not yet robust. If only a small part depends on assumptions, the mechanics-to-outcomes link is more testable.