Using AI to Find Where Clowns End and Juggalos Begin

Source: Deep Learning on Medium

Go to the profile of Nhoral
Photo by Nate Smith

Clowns and Juggalos generally have pretty distinctive looks. So distinctive in fact, that without much work (or rather leaning entirely on the work of others) we can build a super accurate classifier. But that’s not what we’ll do here.

Disclaimer: This article skips over a lot of the core techniques involved in preparing your data and creating a Fast.ai Learner to train a model. If you are interested in how that works, check out my article on using Transfer Learning to Detect Yoga Poses.

No, we will explore a deeper question.

When does Clown become Juggalo and vice versa? When is too hard for even a modern AI classifier to tell them apart?

Clowns That Might Be Juggalos

Following the exact same steps outlined here, we can evaluate images that produce the highest loss. This gives is a good sense of images that were either misclassified or where it is simply not entirely confident in its choice.

The overlap of Scary Clown and Juggalo is definitely high

The listed example is pretty common in our top losses, groups and scary clowns.

Some baseless guesses:

  • Many of our Juggalo photos are from large events, and so crowds tend to make the model less confident about a prediction of clown.
  • Scary clowns tend to have more dark paint around the eyes, which is also a feature of Juggalo make-up.

It’s worth noting that the model classified all but the first image correctly, but was less confident about these choices.

Juggalos That Might Be Clowns

Checking out the other side, we can see where our model gets confused with Juggalo face paint. Some of these images could go either way, so I am not surprised that our model had a hard time.

This is really showing us which Juggalos are thinking outside the box

My takeaway is that red color is generally pretty indicative of high Clownosity in an image.

In Summary

Thanks to the Clowns and Juggalos that put on make-up and allowed my model into their culture and into their hearts.

If any of this is interesting to you, check out course.fast.ai.