How to Evaluate Machine Learning Model Performance in Python?

Original article was published on Artificial Intelligence on Medium

Confusion Matrix

Confusion Matrix is one of the core foundations for many evaluation measures in Machine Learning. Confusion Matrix is an “n-dimensional” matrix for a Classification Model which labels Actual values on the x-axis and the Predicted values on the y-axis.

Generally, for a binary classifier, a confusion matrix is a 2×2-dimensional matrix with 0 as the negative class and 1 as the positive class.

For a good model, the principle diagonal elements of the confusion matrix should be high values and off-diagonal elements should be low values.

Each cell in a confusion matrix plays a very important role in understanding the model’s performance. TN, FN, FP, TP are the names give to each cell based on their actual and predicted values in the cell. Let’s understand them one by one and then we’ll find some amazing relations from them.

Fig 1: Confusion Matrix for a binary classifier.

TN = True Negative (Data which was actually negative and was correctly classified in a negative class by the model).

FN = False Negative (Data which was actually positive but was wrongly classified in a negative class by the model).

FP = False Positive (Data which was actually negative but was wrongly classified in a positive class by the model).

TP = True Positive (Data which was actually positive and was correctly classified in a positive class by the model).

There some amazing relations between these notations which are going to be very useful for our evaluation. Before that let’s see what are actually positive and actual negative values.

Fig 2: Negative and Positive cells of Confusion Matrix

Positive (P): The sum of the False Negative and True Positive values is the actual set of positive values in the data. P = FN+TP

Negative (N): The sum of the True Negative and False Positive values is the actual set of positive values in the data. N = TN+FP

Now that we know the cell notations and the positive and negative notations, there are 4 important measures that are calculated using these notations which are very useful for model performance estimation. They are as follows:

  1. True Positive Rate (TPR): True Positive / Total Positive = TP/P
  2. True Negative Rate (TNR): True Negative/ Total Negative = TN/N
  3. False Positive Rate (FPR): False Positive / Total Negative = FP/N
  4. False Negative Rate (FNR): False Negative/ Total Positive= TN/P

They sound a bit confusing but when we look at the confusion matrix and find these relations pictorially, then they start to give some sort of sense. For a model to perform well “TPR and TNR should be high values” and “FPR and FNR should be low values”.

TPR⬆, TNR⬆, FPR⬇, FNR⬇