What is a Confusion Matrix?

A confusion matrix is a table that is often used to evaluate the performance of a machine learning model. The confusion matrix shows the number of correct and incorrect predictions made by the classification model compared to the actual outcomes.

The matrix is divided into four sections, or quadrants, which represent:

  • True Positives (TP): the number of correct positive predictions
  • False Positives (FP): the number of incorrect positive predictions
  • True Negatives (TN): the number of correct negative predictions
  • False Negatives (FN): the number of incorrect negative predictions

Here is a visual representation of a confusion matrix:

Predicted Positive Predicted Negative
Actual Positive True Positive (TP) False Negative (FN)
Actual Negative False Positive (FP) True Negative (TN)

The goal of any machine learning model is to have as many correct predictions as possible, and as few incorrect predictions as possible. The confusion matrix helps us understand how well the model is performing in terms of both positive and negative predictions.

Calculating Metrics from a Confusion Matrix

From the confusion matrix, we can calculate several metrics that help us evaluate the performance of our model:

  • Accuracy: the proportion of correct predictions out of the total number of predictions (TP + TN / TP + TN + FP + FN).
  • Precision: the proportion of true positive predictions out of the total number of positive predictions (TP / TP + FP). This metric focuses on the correctness of positive predictions.
  • Recall (also known as sensitivity or true positive rate): the proportion of true positive predictions out of the total number of actual positive cases (TP / TP + FN). This metric focuses on the completeness of positive predictions.
  • Specificity (also known as true negative rate): the proportion of true negative predictions out of the total number of actual negative cases (TN / TN + FP). This metric focuses on the correctness of negative predictions.
  • F1 score: the harmonic mean of precision and recall (2 * (precision * recall) / (precision + recall)). This metric balances the trade-off between precision and recall.

 

For Example:

Using the following confusion matrix to calculate Accuracy, precision, recall, specificity and F1 score.

Accuracy: (100+50/100+50+10+5) = 0.909

Precision: (100/100+10) = 0.9

Recall: (100/100+5) = 0.95

Specificity: (50/50+10) = 0.833

F1 Score: (2*(0.9*0.95)/ (0.9+0.95)) = 0.924

Interpreting a Confusion Matrix

Interpreting a confusion matrix can be challenging, especially if you are not familiar with the metrics mentioned above. We want to maximize the number of true positive predictions and minimize the number of false positive and false negative predictions.

In some cases, we may be more interested in certain metrics than others. For example, in medical diagnosis, we may want to maximize recall to minimize false negative predictions, even if it means increasing the number of false positive predictions.

A confusion matrix is a useful tool for evaluating the performance of a machine learning model. It provides a detailed breakdown of the model’s predictions and allows us to calculate several metrics that help us understand how well the model is performing. By analysing the confusion matrix, we can identify areas where the model needs improvement and adjust improve its accuracy, precision, recall, specificity, and F1 score.

The Data School
Author: The Data School