Understanding IOU metric in object detection.

Original article was published on Deep Learning on Medium

Understanding IOU metric in object detection.

Once you build an object detector next thing you want to know is how accurate your model. This is common across all machine learning projects that before implementation, one should evaluate the model. In the case of regression problems we have a metric like Mean Square Error (MSE), similarly, for classification one can use confusion matrix,f1 score, etc; refer to this blog for detail.

Now coming back to our problem, the task of object detection or output of object detection is nothing but draw a bounding box with a certain probability.

Sample Output: Assume we are detecting batsman

The bounding box is a set of coordinates to use to localize image, it varies for COCO and Pascal-VOC datasets.

  • COCO: bbox is (top left x position, top left y position, width, height)
  • Pascal-VOC:bbox is (xmin-top left, ymin-top left,xmax-bottom right, ymax-bottom right)

Further details about these datasets can be read here.

Red Box: Bounding box from prediction.

Black Box: Actual bounding box or ground truth bounding box.

Unlike other metrics such as accuracy ,f1 score , here we use the concept of similarity instead of point match of bbox. One such measure of similarity is the Jaccard index or in the colloquial language of computer vision, we refer it as intersection over union.

Suppose we have two sets S1 and S2, Jaccard index J is defined as

J(S1, S2) = |S1 ∩ S2|/ |S1 U S2|

IoU = Area of INTERSECTION /Area of UNION.

  • IoU score ≥0.5 is considered as good.

Python Implementation

https://gist.github.com/ppushp7/a3bdafb19acc9ed20c6d0b4ad1f15b48

box1 = (21, 11, 14, 13)
box2 = (23, 12, 13, 14)
print("iou = " + str(iou(box1, box2)))

Output : iou = 0.417

Some other applications of the IoU metric.

Non-max suppression

Once you have fixed a threshold all bounding box above that threshold is filtered, but we want a single bbox for each object in the image.

How Non-max suppression works?

Given set of predicted bbox and probability/score along with threshold, one needs to come up with a single bbox.

A: Set of all predicted bbox with probability

B: initially empty set

  • pick the one with the highest score and push into the set B
  • drop that one from A.
  • calculate IoU of all others with the one picked, remove all predicted box from A whose IoU ≥ thresholds.
  • repeat steps 2 and 3 untill there is no more bbox left in A.