R Squared: define the concept before backtesting
R Squared (often written R²) is a statistic that describes how much of the variation in a measured outcome can be explained by a model or relationship. In backtesting, the key responsibility is not to treat R² as a trading signal by itself, but to define exactly what you are regressing on what, and what the “explained outcome” means in your use case.
A responsible workflow starts by writing down:
- the dependent variable (the outcome you measure)
- the independent variable(s) (the feature(s) or indicator values you use)
- the method used to compute the relationship (for example, a regression or a correlation-like approach)
- the time alignment rule (which timestamp/price feeds into which period’s outcome)
If you cannot state these items precisely, the backtest results are not independently verifiable.
Mechanics: decide data, assumptions, and costs
Backtests are only meaningful when the data preparation rules are explicit. Common responsibilities include:
Data definition and sampling
Choose a consistent time grid (for example, every bar close) and specify how you handle missing observations. Define the lookback window used to compute R². If you use a rolling window, specify its length and whether it updates on every bar or only at certain times.
Time alignment and leakage controls
A common failure mode is look-ahead bias: using information that would not have been known at the decision time. Make an alignment rule such as “indicator value computed using data up to time t is evaluated against the outcome from t to t+1.” Then enforce it in code and by spot-checking dates.
Costs and execution model
Even if you only test a relationship statistic, many workflows eventually translate it into decisions. If you do, you must model material frictions:
- bid/ask spread assumptions
- commissions or fees
- slippage from market impact or order filling
A responsible backtest treats costs as part of the data-to-outcome pipeline, not as an afterthought. Otherwise, R² may look strong while any practical, cost-aware outcome would be weaker or inconsistent.
Evidence: use bias controls and out-of-sample checks
To verify that your “R² effect” is not an artifact, use separation of data and strict evaluation.
Bias controls
At minimum, control the major sources of bias:
- look-ahead bias (time alignment)
- selection bias (choosing periods after seeing results)
- overfitting bias (tuning parameters to maximize in-sample performance)
A practical approach is to treat any parameter choices (window length, preprocessing rules, thresholds) as fixed before the final evaluation step.
Out-of-sample testing
R² computed on the same data used to define the relationship can overstate reliability. Use:
- a training (in-sample) period to set any choices
- a validation period to confirm you are not just fitting noise
- a final out-of-sample period for the most conservative assessment
If the relationship changes materially across periods, that is an important finding.
One material limitation and why it matters
A major failure mode is that historical explanatory relationships do not guarantee future stability. Market regimes can shift, and statistical fit can decay when conditions change, especially when costs, liquidity, or execution quality differ from the assumptions.
Limitations and risks you should be able to explain
A responsible backtest should include a clear limitations section that addresses:
- Non-stationarity: relationships can vary over time.
- Model specification sensitivity: small changes in inputs or alignment can change R².
- Overfitting risk: high in-sample fit can still produce weak real-world results.
- Cost sensitivity: ignoring spread and slippage can turn an apparent relationship into an illusion.
Also remember: R² is a measure of fit to an outcome definition. If your outcome variable is poorly chosen or inconsistent with real decision opportunities, the statistic can mislead.
Verification: what you can independently check next
To independently verify a responsible R² backtest, you should be able to show others:
- the exact definition of the dependent and independent variables
- the time alignment rule and windowing method
- the cost and execution assumptions used in the pipeline
- the data split strategy (training/validation/out-of-sample) and which parameters were tuned
- at least one failure mode you tested for (for example, look-ahead bias via a known date-shift check)