R-Squared

In one sentence: R-squared is a score that says how much of the ups and downs in the data a model predicts correctly. A score of 1 is perfect, and 0 means the model is no better than always guessing the average.

Read it out loud: “R squared equals 1 minus the ratio of SS res to SS tot. R squared is the coefficient of determination. SS res is the sum, over every data point i, of the squared gap between the actual value y sub i and the prediction y hat sub i, read as y hat i. SS tot is the sum, over every data point, of the squared gap between the actual value and the average, y bar. The capital sigma means add up. The letter i labels the data points, so point 1, point 2, and so on.”

Before you start

The ideas and signs this page uses, explained from zero. For powers of ten, Greek letters and other notation in general, see Math basics.

Data points, actual values and predictions

A data point is one row of data, such as one student’s quiz result. Its actual value, written $y_i$, is the number that was measured, such as 7 points. A model makes a prediction for each point, written $\hat y_i$ and read “y hat i”, which is its guess for that point. The small letter $i$ is a label that tells the points apart: point 1, point 2, and so on.

The average, written $\bar y$

The average, or mean, is the sum of all the values divided by how many values there are. The bar over the $y$, written $\bar y$ and read “y bar”, means the average of the $y$ values. For the scores 3, 5, 7 and 9, the average is $\frac{3 + 5 + 7 + 9}{4} = 6$.

The sum sign and the index $i$

The capital Greek letter sigma, $\sum$, means add up. $\sum_i$ means add over every data point $i$, from the first to the last. So $\sum_i (y_i - \bar y)^2$ means: take each point’s actual value minus the average, square that number, and add all the squares together.

Squares and why they remove signs

A small raised 2 means the number is multiplied by itself, so $3^2 = 9$. Squaring also removes a minus sign: $(-3)^2 = 9$, the same as $3^2$. Squares make big misses count more than small ones. A miss of 2 gives a square of 4, while a miss of 1 gives a square of only 1.

Fractions as a share

A fraction divides one number by another. The fraction $\frac{0.75}{20}$ means 0.75 divided by 20, which is 0.0375. When one amount is a part of a total, the fraction is its share of that total. A share of 0.0375 means 3.75 percent of the total.

Regression, a model that predicts a number

A regression model predicts a number, such as a score or a price, from other information. A straight-line model draws the line that fits the data points best. R-squared checks how closely such a model’s predictions follow the actual numbers. It is not used for predicting categories, such as spam or not spam.

What each symbol means

The formula piece by piece

$y_i - \hat y_i$

One miss, which is called a residual. It is the actual value minus the prediction. It is positive when the model guessed too low, and negative when the model guessed too high.

$(y_i - \hat y_i)^2$

The squared miss. Squaring makes every miss positive, so misses in opposite directions do not cancel. A miss of 2 points counts four times as much as a miss of 1 point.

$SS_{res} = \sum_i (y_i - \hat y_i)^2$

The total of all the squared misses. A small total means the predictions are close to the actual values. It is zero only when every prediction is exact.

$SS_{tot} = \sum_i (y_i - \bar y)^2$

The total squared distance of the actual values from their average. This is the error of the simplest guess, which predicts the average for every point. It uses the same squaring idea, with the average in place of the prediction.

$\frac{SS_{res}}{SS_{tot}}$

The share of the spread that the model fails to explain. It compares the model’s squared misses with the squared misses of the simplest guess. It is 0 for a perfect model and 1 for a model that is no better than the average.

$R^2 = 1 - \frac{SS_{res}}{SS_{tot}}$

R-squared equals 1 minus the share of the spread that the model fails to explain. What is left over is the share the model does explain.

$R^2 = r^2$

A second form that holds only for a straight-line fit with one input. Here $r$ is the correlation coefficient, a number from $-1$ to $1$ that says how closely two quantities move together. In that one case, squaring $r$ gives the same value as R-squared.

What the formula is doing, and why

The input is a list of actual values and the model’s predictions for them. The output is one number, R-squared, which compares the model with the simplest possible guess. That guess is the average of the actual values, used for every point. R-squared asks how much of the error of that simple guess the model removed.

Why square the misses? A plain sum of misses would let positive and negative misses cancel, so a poor model could look perfect. Squaring makes every miss count as positive. It also punishes big misses more than small ones. The same squaring is used for the simplest guess, so the two totals can be compared fairly.

Why use the average as the baseline? The average is the one constant guess with the smallest total of squared misses, so it is the strongest simple guess a model must beat. The ratio $\frac{SS_{res}}{SS_{tot}}$ is the share of the simple guess’s error that is still left over. Subtracting it from 1 gives the share the model removed. A perfect model has $SS_{res} = 0$ and scores 1. A model that does no better than the average scores 0.

When can it go below zero? When the predictions are worse than always guessing the average, $SS_{res}$ can be bigger than $SS_{tot}$, and R-squared becomes negative. This can happen with a badly chosen line, or with a model tested on new data it was not built on. For a least-squares line with an intercept, measured on the same data it was fitted to, the value sits between 0 and 1.

When it applies. R-squared suits predictions of numbers. It does not work for categories, such as spam or not spam. It also says nothing about cause. A high score means the predictions follow the data closely, not that one thing causes another.

An everyday picture

Think of guessing the heights of everyone in a class. Your baseline is to guess the class average for every student. R-squared asks how much of that baseline error your smarter guesses removed. Remove all of it and you score 1. Remove none and you score 0. Make things worse than the average and the score drops below zero.

Worked example: Predicting four quiz scores

Four students scored $y = (3,\ 5,\ 7,\ 9)$ points on a quiz. A model predicted $\hat y = (2.5,\ 5.5,\ 7,\ 9.5)$ points for them. We want to know how much of the spread in the scores the model explains.

1. Find the average score. Add the four actual scores and divide by 4: $\bar y = \frac{3 + 5 + 7 + 9}{4} = 6$ points. This average is the simplest possible guess, the same 6 points for every student.
2. Measure the total spread around the average. Each score’s distance from 6 is $-3$, $-1$, $1$ and $3$. Squaring gives $9$, $1$, $1$ and $9$, which add to $SS_{tot} = 20$ squared points. This is the error the simplest guess makes.
3. Measure the model’s squared misses. The misses are $3 - 2.5 = 0.5$, $5 - 5.5 = -0.5$, $7 - 7 = 0$ and $9 - 9.5 = -0.5$. Squaring gives $0.25$, $0.25$, $0$ and $0.25$, which add to $SS_{res} = 0.75$ squared points.
4. Compare the two errors. The model’s error is a share of the simplest guess’s error: $\frac{0.75}{20} = 0.0375$. That means the model still misses 3.75 percent of the spread.
5. Subtract the share from 1. $R^2 = 1 - 0.0375 = 0.9625$. The model explains about 96 percent of the spread in the scores. It still misses a little, but almost all of the variation is captured.

Answer: R-squared is 0.9625, about 0.96. The model explains about 96 percent of the spread in the scores. It still misses a little, but almost all of the variation is captured.

The numbers in this example are recomputed by script every time the site is built.

Common mistakes

• Reading R-squared as proof that the model is right, or that one thing causes another. It only measures how closely the predictions follow the data.
• Comparing R-squared values across datasets with different outputs. The total spread changes with the data, so the scores are not on a shared scale.
• Adding up plain errors instead of squared errors. Positive and negative misses cancel, so the total can be zero even when the predictions are poor.
• Expecting R-squared to stay between 0 and 1 for every model. A model worse than the average, or one tested on new data, can give a negative score.

Where you will meet it

• Reporting how well a straight-line or multiple regression fits its data, in statistics textbooks and in the trendline displays of spreadsheets.
• Scoring regression models in machine learning libraries, such as the r2_score function in scikit-learn.
• Economics and the social sciences, where a model’s R-squared summarizes how much of the variation in the data it accounts for.
• Checking how closely a calibration curve follows reference measurements in a laboratory.

Questions people ask

Is R-squared the same as correlation?

For a straight-line fit with one input, R-squared equals the square of the correlation coefficient $r$. The correlation coefficient is a number from $-1$ to $1$ that says how closely two quantities move together. With more inputs, or with other kinds of model, that shortcut no longer holds.

What counts as a good R-squared?

It depends on the field. Noisy data, such as human behavior, often gives much lower values than a careful physics measurement. Compare it with other models on the same data rather than with a fixed cutoff.

Why can R-squared be negative?

Because $SS_{res}$ can be bigger than $SS_{tot}$. That happens when the predictions are worse than always guessing the average.

Does adding more inputs always raise R-squared?

On the training data it never goes down when you add an input, even a useless one. Training data is the set of examples used to fit the model. Check the score on new data as well before trusting a bigger model.

Related formulas

• Mean Squared Error (MSE): Mean squared error measures how wrong a model’s guesses are by squaring each miss, so big misses count much more, and then averaging all of the squared misses.
• Gradient Descent: Gradient descent nudges a model’s settings a small step in the direction that makes the error smaller, then repeats that step again and again until the error stops falling.
• Z-Score Standardization: A z-score tells how far a value is from the average, counted in typical steps of spread, so features measured in different units can be compared fairly.

Sources

1. Coefficient of determination, Wikipedia
2. Linear regression, Wikipedia