Precision, Recall, and Relevance


How relevant is your response?
A simple question to ask, but figuring out the answer can easily lead you down a rabbit-hole. Before we go down that though, let’s first take a look at what “Relevant” actually means.

Relevance is the concept of one topic being connected to another topic in a way that makes it useful to consider the second topic when considering the first.”
 — Wiki

Or, to put it differently, Relevance is all about connections, between ideas, concepts, topics, communications, whatever. In short, when we want to know how relevant something is, what we’re actually asking is how connected it is to something else you are interested in.

Relevance becomes, well, relevant when we start thinking about accuracy in responses. For example, let’s say we’re talking about cats (because, cats!). If you ask me how many pictures of cats I’ve tweeted, and I respond 211 (note: not an accurate number), then how do you measure my accuracy? 
Well, there are a number of ways, but in this case, let’s focus on precision, and recall.

Precision is a measure of the quality of my response, i.e., how much of my response was correct. If you looked at the 211 tweets, and said “Hey, 200 of these are cats, but 11 of them are pictures of pizza!”, then my precision was 200/211. (Only 200 of my 211 were cats)
In short, think of precision as a count of how many of the selected items were relevant (in math — Precision = True Positives / (True Positives + False Positives)

Recall, OTOH, is a measure of quantity in my response, i.e., how close was I to the actual answer. If you looked at all of my tweets, and found 727 more pictures of cats (that I, for some reason, hadn’t counted 🙄), then my recall was 200/(727+200) = 200/927 (200 of my 211 were cats, and there were 727 other cats that I didn’t count)
In short, think of recall as a count of how many of the relevant items were selected (in math — Recall = True Positives / (True Positives + False Negatives)

Precision and Recall are ridiculously relevant in Machine Learning (after all, it used to be huge in pattern recognition…). For much more about this, take a look at the Wiki page, and then just google around…

(This article also appears on my blog)

Source: Deep Learning on Medium