False Positives vs. False Negatives

Original article was published on Artificial Intelligence on Medium

Example of Committing Errors and Consequences

Let’s say you are doing an experiment in trying to classify if an object is an egg or not. You are given a dozens of real eggs and dozens of egg shaped rocks. Now the kicker here is you can only classify them visually without touching them.

So let’s fast forward through this experiment and arrive to the section where you created a machine that can classify an egg with about 90% accuracy. This is where the errors may potentially come in. The consequences associated with each error is dependent on the problem.

Photo by Natalie Rhea Riggs on Unsplash

Egging a House

For example, a Type I Error or False Positive would be if our machine were to classify an egg shaped rock as an egg. This could be worse than a Type II error in several ways. What if we were a bunch of kids looking to egg a house, and we ended up throwing a rock and breaking a window. Something that would have just cost time and water to clean now could have potentially injured someone and turned into a serious act of vandalism.

A Type II Error of False Negative in this instance would be if our machine classified an egg as not an egg. In our example, this would not be much of an issue because we would just not pick this non-egg. Now, as a bunch of kids, we would not use this non-egg because we want to egg a house without destroying anything.

Which is Worse?

In this example, we can see that a Type I error is more costly and consequential than a Type II error. Here, a Type I error is worse than a Type II. We would hope to not misclassify an egg shaped rock as an egg because of the purpose we are using these eggs for.

Now if we changed the purpose of these eggs, then the worse type of error we can commit can change also. How would a Type II error be worse than a Type I?

Golden Eggs

Photo by Sharon McCutcheon on Unsplash

How about if we wanted to find as many eggs as possible because each egg is extremely valuable. Maybe the eggs contain gold or something of equal value. In this case, if our machine misclassifies an egg as not an egg, then we potentially missed out on some gold. However, even if our machine misclassified a rock as an egg, then the mistake would not be as costly as in the egging example.