Classification Accuracy

In one sentence: Accuracy is the share of a model’s guesses that were right, out of all the guesses it made.

$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$

Read it out loud: “Accuracy equals T P plus T N, divided by T P plus T N plus F P plus F N. The letters stand for counts of examples: T P is the number of true positives, T N the number of true negatives, F P the number of false positives, and F N the number of false negatives. Each count is a whole number of examples.”

Before you start

The ideas and signs this page uses, explained from zero. For powers of ten, Greek letters and other notation in general, see Math basics.

Class, positive and negative: A class is a group that a model can put an item into, such as spam or not spam. In a yes-or-no classifier, the positive class is the one you are looking for, here spam, and the negative class is everything else. The words positive and negative name the classes. They do not mean that an answer is good or bad.
Counts and the confusion matrix: A count is a whole number of items, such as 15 spam emails. The four counts $TP$ , $TN$ , $FP$ and $FN$ come from comparing each prediction with the true label. Laying them out in a two-by-two table gives a confusion matrix, a standard grid that shows where a model gets confused between the two classes.
Share, fraction and percent: A share is one part of a whole, written as a fraction between 0 and 1. Ninety correct answers out of 100 gives $\frac{90}{100} = 0.90$ , and 0.90 is the same as 90%. A percent is a count out of 100, so 90% means 90 out of every 100.
Imbalanced classes: A dataset is imbalanced when one class has many more examples than the other, such as 99 normal emails for every 1 spam email. In an imbalanced dataset a model can reach a high accuracy by always guessing the big class, so accuracy needs a second check that looks at each class.
Training data and test data: A model is trained on examples it can learn from. A test set is a separate group of examples that the model has never seen, and it is used to measure how well the model works on new data. Accuracy measured on the training examples is usually too optimistic, because the model has already seen those answers.

What each symbol means

Symbol	Meaning
$\text{Accuracy}$	The share of all predictions that were correct. It is a number from 0 to 1 with no unit, often written as a percentage. An accuracy of 0.9 means that 90 out of every 100 predictions were right.
$TP$	True positives: items the model said were positive and really were positive, such as a spam email correctly flagged. It is a count of examples with no unit, and it can never be negative.
$TN$	True negatives: items the model said were negative and really were negative, such as a normal email correctly left in the inbox. It is a count of examples with no unit, and it is the largest group when most examples are normal.
$FP$	False positives: the model said positive but the answer was negative, such as a normal email wrongly flagged as spam. It is a count of examples with no unit, and it is also called a false alarm.
$FN$	False negatives: the model said negative but the answer was positive, such as a spam email that reached the inbox. It is a count of examples with no unit, and it is also called a miss.

The formula piece by piece

$TP + TN$: The correct answers. The model was right in two ways: it caught the true positives and it correctly passed the true negatives. Adding them gives the count of right answers, which is the top of the accuracy fraction.
$TP + TN + FP + FN$: Every answer the model gave, right or wrong. The four boxes of the confusion matrix together hold every example that was tested, so this total is the number of examples, and it is the bottom of the accuracy fraction.
$\frac{TP + TN}{TP + TN + FP + FN}$: The share of right answers: the correct answers divided by all the answers. A fraction like this is always between 0 and 1, because its top counts only part of its bottom.
$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$: Accuracy is the number of correct predictions divided by the total number of predictions, so it is a share between 0 and 1 that says how often the model was right.

What the formula is doing, and why

A classifier puts each example into a class, such as spam or not spam. For a yes-or-no question, every prediction lands in one of four boxes. The positive class is the one you are looking for, here spam. Counting how many predictions land in each box gives $TP$ , $TN$ , $FP$ and $FN$ . Together they form the confusion matrix.

Why the top and bottom look like this. The top, $TP + TN$ , adds the two kinds of correct answer, because both are right. The bottom adds all four boxes, because every answer the model gave counts as one attempt, right or wrong. Dividing the right answers by all answers gives the share that were right, which is what accuracy means.

Why the result stays between 0 and 1. The top can never be bigger than the bottom, because the top is only the correct part of the bottom. A value of 0.9 means 90% of the predictions were right. Accuracy gives every mistake the same weight: a missed spam email and a wrongly flagged normal email each count as one error, even though their costs in real life may differ.

When accuracy misleads. The equal weighting is the weak point when one class is much bigger than the other. If 99 of every 100 emails are normal, a model that always answers not spam is right 99% of the time, yet it catches no spam at all. Accuracy looks excellent while the model fails at its real job. In that case, check the errors for each class separately, for example with precision and recall.

An everyday picture

Think of a spelling test with 100 words. If you spell 90 words correctly, your score is 90 out of 100, and accuracy works the same way. The difference is that a classifier’s test set may hold mostly one kind of case, so a high score can hide a failure on the rare kind.

Worked example: A spam filter that checks 100 emails

A made-up test set of 100 emails: 20 are spam and 80 are not spam. The filter catches $TP = 15$ spam emails, misses $FN = 5$ spam emails, keeps $TN = 75$ normal emails in the inbox and wrongly flags $FP = 5$ normal emails. All these numbers are invented for the example, and each count is a number of emails with no unit.

Check that the counts add up. Before dividing, confirm that the four boxes cover all 100 emails. The spam row is $15 + 5 = 20$ and the normal row is $75 + 5 = 80$ , and together they make 100. This check matters, because a mistake in one count would quietly change the answer.
Count the correct predictions. The correct predictions are the true positives and the true negatives: the spam that was caught and the normal emails that were kept. So $TP + TN = 15 + 75 = 90$ emails were classified correctly.
Count all the predictions. The bottom adds all four boxes, right and wrong: $15 + 75 + 5 + 5 = 100$ . Since 100 emails were tested, every email was counted exactly once, which matches the size of the test set.
Divide to get accuracy. Accuracy is the share that was right: $\text{Accuracy} = \frac{90}{100} = 0.90$ . As a percent, the filter is 90% accurate on this test set. That sounds strong, so the next step checks whether the score really means something.
Compare with a model that always says not spam. Such a model answers normal for all 100 emails. It gets the 80 normal emails right and the 20 spam emails wrong, so its accuracy is $\frac{80}{100} = 0.80$ . It catches zero spam, yet it still scores 80%, which shows that accuracy alone can hide a useless model.

Answer: The filter’s accuracy is 90%. That sounds strong, but a model that always answers not spam reaches 80% and catches no spam at all. When one class is much more common, accuracy alone can hide the errors that matter.

The numbers in this example are recomputed by script every time the site is built.

Common mistakes

Reading 99% accuracy as success without checking the class counts. If 99% of the data belongs to one class, a model that always guesses that class also scores 99%.
Mixing up false positives and false negatives. They are different errors: a false positive hides a good email from you, and a false negative lets spam through.
Measuring accuracy only on the training data. Scores on examples the model has already learned from are too optimistic. Use a separate test set.
Dividing by the wrong total. The bottom of the fraction must count every prediction, not only the correct ones and not only the positive examples.

Where you will meet it

Quick first checks of image classifiers, such as sorting photos of handwritten digits from 0 to 9, where the classes are fairly balanced.
Benchmark leaderboards for image classification, where top-1 accuracy is the share of test images whose single best predicted label is correct.
Summary scores for spam filters and other yes-or-no tools, usually shown next to precision and recall.
Comparison tables in research papers, where each model’s accuracy is reported on the same test set so the models can be compared fairly.

Questions people ask

What is a confusion matrix?

A table that counts the four outcomes: true positives, true negatives, false positives and false negatives. Usually the actual classes are the rows and the predicted classes are the columns. The correct answers sit on the diagonal, so accuracy is the diagonal added up and divided by the total.

What counts as a good accuracy?

There is no universal number. Compare the score with a baseline, such as always guessing the most common class. An accuracy of 80% means little if that baseline already reaches 80%.

Is accuracy the same as the error rate?

They are linked. The error rate is the share of wrong answers, $\frac{FP + FN}{TP + TN + FP + FN}$ , and it always equals $1 - \text{Accuracy}$ . In the example, the error rate is 10 out of 100, which is 0.10.

When should I avoid relying on accuracy?

When one class is rare, or when the two kinds of mistake have very different costs. In medical screening, for example, a missed case can matter far more than a false alarm, so accuracy alone would be a poor summary.

Precision and Recall: Precision is the share of the items a model flagged that were really right, and recall is the share of the real cases that the model managed to find.
F1 Score: The F1 score is one number from 0 to 1 that combines precision and recall, and it stays low unless both of them are reasonably high.
Softmax: Softmax takes a list of scores, which can be any numbers at all, and turns them into percentages that add up to 100 percent, so the biggest score gets the biggest share.