Define R Squared before collecting data
R Squared (often written R²) is a summary number that compares how well a model’s predictions match observed outcomes. The usual idea is: how much of the variation in the observed dependent variable is explained by the model, relative to a baseline that predicts something like the mean.
To assess R Squared, you first need to know the exact definition being used (for example, whether it is computed from the total sum of squares and the residual sum of squares in the standard way, and whether any “adjusted” variant is used). Different implementations can report different numbers even when the underlying data is similar.
Data inputs you need
At minimum, you need all information needed to reproduce the dependent side of the calculation.
- Observed dependent variable series
- The time-indexed observations that form the y-values.
- The sampling frequency (e.g., every tick, every minute, daily), including the units.
- The time alignment rules (when each y-value is considered “corresponding” to predictors).
- Predicted values or the fitted model You have two options:
- Predicted values: The model’s predicted y-hat values for the same time points, with the same ordering.
- Or, the model inputs needed to generate y-hat: the predictor series (x-values), the estimation method, and the parameters produced by fitting.
- The baseline used in the denominator Most R² formulas use a baseline that reflects variation around the mean of the observed dependent variable. To assess R² correctly, you must know what baseline is used and whether the mean is computed on the same dataset split as the residuals.
Provenance and preprocessing checks
R² is not just a formula; it depends on what went into the series.
Check and record:
- Data provenance: where the data came from (feed or source), and how it was obtained.
- Data filtering rules: whether observations were removed due to errors, holidays, outliers, or low liquidity.
- Handling missing values: whether gaps were dropped, forward-filled, interpolated, or otherwise treated.
- Transformations: whether any scaling, log transforms, differencing, or normalization was applied, and whether it was consistent across train and evaluation periods.
Material risk: if x and y are preprocessed differently, or if timestamps are misaligned, R² can look higher or lower for reasons unrelated to genuine explanatory power.
Timeliness and split assumptions
To assess whether an R² value is meaningful, you need to know how time was used.
You should document:
- Whether R² was computed on the same data used to fit the model or on held-out data.
- The specific time window used for fitting versus evaluation.
- Whether a rolling or expanding window approach was used (the approach changes the meaning of the reported result).
A common failure mode is treating in-sample R² as if it describes future relationships. Historical relationships can change when market behavior shifts, so the same method may yield different R² on later data.
Evidence or example: what you can verify independently
An independent verification checklist should include:
- Confirm the exact y series used (timestamps, units, and filtering).
- Confirm whether the reported value used y-hat predictions or recomputed predictions from stored model parameters.
- Confirm the exact R² definition (and whether adjusted R² was used).
- Recompute R² from the provided numbers (y, y-hat, residuals) using the stated formula and the same dataset split.
If any of these are missing, the reported R² cannot be fully audited, because many implementation details change the result.
Limitations and risks (material failure modes)
At least one limitation to watch for:
- Model dependence: R² reflects fit under a particular model form. If the relationship is nonlinear or changes over time, the same R² can be misleading.
Other important limitations:
- Non-stationarity: if the data-generating process evolves, a high R² in one period may not hold later.
- Overfitting: if the model is too flexible, it can match noise, inflating in-sample R².
- Sensitivity to noise and rare events: noisy measurements can cause unstable R² across time windows.
- Misleading comparisons: R² values can be difficult to compare across models if preprocessing, feature sets, or evaluation splits differ.