Direct answer: what is R squared?
R squared (often written R²) is a number that describes how much variation in an observed target (the dependent variable) is accounted for by a set of fitted predictions (the model’s output). In its most common form, R² is computed from two quantities:
- Total variation in the observed data
- Residual (unexplained) variation after using the fitted predictions
Interpretation, in plain terms: if residual variation is small compared with total variation, R² is higher; if residual variation is large, R² is lower.
Mechanism: the formula and every required input
1) Define the data
To calculate R² you need a collection of observed values and corresponding fitted (predicted) values.
- Observed values: (y_1, y_2, \dots, y_n)
- Fitted (predicted) values: (\hat{y}_1, \hat{y}_2, \dots, \hat{y}_n)
- The sample mean of observed values: (\bar{y} = \frac{1}{n}\sum_{i=1}^{n} y_i)
Where (n) is the number of data points you are evaluating.
2) Compute the sums of squares
R² uses sums of squares based on differences from means and differences from fitted predictions.
-
Total sum of squares (variation around the mean): [ \mathrm{SST} = \sum_{i=1}^{n} (y_i - \bar{y})^2 ]
-
Residual sum of squares (variation left after fitting): [ \mathrm{SSE} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 ]
3) Calculate R squared
In the most common definition: [ R^2 = 1 - \frac{\mathrm{SSE}}{\mathrm{SST}} ]
This requires (\mathrm{SST} > 0). If all observed (y_i) are identical, then (\mathrm{SST}=0) and R² is not meaningfully defined in this form.
4) Key practical choice: which data are (\hat{y}) computed on?
The formula itself does not specify whether (\hat{y}) comes from:
- the same data used to fit the model (in-sample), or
- separate data not used during fitting (out-of-sample)
Using in-sample fitted values often inflates performance for flexible models. Using out-of-sample predictions generally gives a more realistic sense of how well the relationship generalizes, but it can lower R² even when the model is directionally useful.
Evidence or example: a concrete calculation (with explicit assumptions)
Assume you have (n=3) observed values (y) and a model that produces fitted values (\hat{y}) for the same three points.
Let:
- (y = [2,\ 4,\ 6])
- (\hat{y} = [2,\ 5,\ 5])
Step 1: compute the mean [ \bar{y} = (2+4+6)/3 = 4 ]
Step 2: compute (\mathrm{SST}) [ \mathrm{SST} = (2-4)^2 + (4-4)^2 + (6-4)^2 = 4 + 0 + 4 = 8 ]
Step 3: compute (\mathrm{SSE}) [ \mathrm{SSE} = (2-2)^2 + (4-5)^2 + (6-5)^2 = 0 + 1 + 1 = 2 ]
Step 4: compute R² [ R^2 = 1 - 2/8 = 1 - 0.25 = 0.75 ]
Under this calculation, 0.75 indicates that residual variation is one quarter of the total variation (for this evaluation setup).
Limitations and failure modes: what R squared can get wrong
R² is widely used, but it is not a guarantee of usefulness. Several material limitations can cause it to mislead.
1) Overfitting when evaluated in-sample
If a model is flexible and you compute R² using fitted values on the same dataset it learned from, the residual errors may be artificially small. This can produce a high R² even when the model does not capture a stable relationship.
2) Nonlinearity mismatch
R² measures how well predicted values match observed values through residuals. It does not ensure that the underlying relationship has a particular structure (for example, linearity) unless your modeling approach enforces that structure.
A model can show a moderate or high R² while still being inappropriate for extrapolation beyond the data range, because R² is descriptive for the evaluation set, not a universal law.
3) Scale and preprocessing effects
Because SST and SSE depend on the scale of (y), preprocessing steps (such as differencing, normalization, or transformations) can change R². Two analysts working with different transformations can end up with different R² values even if the “quality” of fit is comparable.
4) R² can be undefined or unintuitive
- If (\mathrm{SST}=0), the formula breaks.
- Depending on how predictions are produced and whether you use alternative definitions (such as adjusted variants), the value can be less intuitive than “percentage explained.”
5) Relationship vs. causation
R² quantifies how closely predictions track observations. It does not establish why the relationship exists. Even with a high R², the association could reflect shared drivers, data artifacts, or coincidental patterns.
Verification and next question: how to independently check a reported R squared
If you see a reported R², you can verify the calculation by checking these facts, without relying on claims about future performance:
- Confirm the evaluation dataset size (n).
- Obtain the observed values (y_i).
- Identify the fitted/predicted values (\hat{y}_i) used to compute residual errors.
- Recompute (\bar{y}), SST, SSE, and then (R^2 = 1 - \mathrm{SSE}/\mathrm{SST}).
If the provider or calculator does not state which (\hat{y}) are used (in-sample vs out-of-sample) and what preprocessing was applied, the reported R² is still mathematically defined, but the comparison across setups becomes uncertain.
A useful follow-up question is how R² differs when computed for related notions (for example, using different evaluation splits or modeling forms), because those choices change what R² actually measures.