Create and Deploy your Meme Detector from scratch : A step-by-step guide

Source: Deep Learning on Medium

Create and Deploy your Meme Detector from scratch : A step-by-step guide

Introduction

In this tutorial, you will learn how to build your own Meme Detector from scratch like this one. By the end of the tutorial, you will be able to:

  • Create your own dataset using Google Image.
  • Choose and retrain the dense layers of a CNN architecture.
  • Deploy your model as a web app to Render.com

Libraries and major dependencies

  1. Google Colab for GPU usage
  2. Fastai v 1.0.52
  3. PyTorch v1

Fastai is an amazing library built on top of PyTorch to make deep learning more intuitive and make it require less lines of code. For more information about it you should go to their documentation website : https://docs.fast.ai/

1. Getting the Data

Before getting to the data collection process, let’s get first define what memes are. According to Wikipedia, memes are defined as follows:

An Internet meme, more commonly known as simply a meme (/miːm/ MEEM) is a type of meme whereby a piece of media, traditionally but not exclusively combining image macros with a concept or catchphrase, is spread — often through social media platforms — via the Internet.

So basically, internet memes are images accompanied by funny text.

Before doing anything, you need to set-up google Colab environment:

  • Installing Fastai and its dependencies
!curl -s https://course.fast.ai/setup/colab | bash
from google.colab import drive
drive.mount('/content/gdrive', force_remount=True)
  • Create the working directory
root_dir = "/content/gdrive/My Drive/"
base_dir = root_dir + 'fastai-v3/data/MEMES/'
from fastai.vision import *
path = Path(base_dir)
path.mkdir(parents=True, exist_ok=True)

Now that everything is set-up you will build your memes image dataset

Collecting memes

Go to google image and search for the keyword “meme” and scroll down until you see the “Show more results” button.

Now, you have to run some Javascript code in the browser which will save the URLs of all the images you want for our dataset.

Depending on the browser / OS different shortcuts are there to take you to the console: In Google Chrome press CtrlShiftj on Windows/Linux and CmdOptj on macOS, and a small window the javascript ‘Console’ will appear. In Firefox press CtrlShiftk on Windows/Linux or CmdOptk on macOS. That is where you will paste the JavaScript commands.

Before running the following commands, you should disable ad blocking extensions (uBlock, AdBlockPlus etc.) in Chrome.

After that, run the following command in the console

urls=Array.from(document.querySelectorAll('.rg_i')).map(el=> el.hasAttribute('data-src')?el.getAttribute('data-src'):el.getAttribute('data-iurl'));
window.open('data:text/csv;charset=utf-8,' + escape(urls.join('\n')));

This will create a text file with a list of URLs to the images in the search results.

You may want to repeat the process with related keywords such as “Anime memes”, “World war memes”, “Black and white memes”… and add the links to the same final list.

You will be naming it “url_memes.txt”.

Now, to download the images, you will first create a folder for the meme class named meme. Then you will download the images from the URL using the Fastai built-in function called download_images.

folder = 'meme'
file = 'url_memes.txt'
path = Path(base_dir)
dest = path/folder
dest.mkdir(parents=True, exist_ok=True)
download_images(path/file, dest, max_pics=300)

Creating the not memes set

The same google search process is repeated for the not meme class using the following keywords : picture, image, selfie, face, photo, wallpaper …

The final URL list is stored in a file called “url_not_memes.txt”

Now, you should run the following script

folder = 'not_meme'
file = 'url_not_memes.txt'
path = Path(base_dir)
dest = path/folder
dest.mkdir(parents=True, exist_ok=True)
download_images(path/file, dest, max_pics=300)

It will create the not_meme folder and download the images there for you.

Congratulations !! you created your first image dataset and that’s HUGE ! you should be proud of yourself.

2. Training the model

You should be able your data using the built-in fastai ImageDataBunch :

np.random.seed(42)
data = ImageDataBunch.from_folder(path, train=".", valid_pct=0.2, ds_tfms=get_transforms(), size=224, num_workers=4).normalize(imagenet_stats)

Basically, the from_folder method takes the following arguments:

  • path : Path to image dataset
  • ds_tfms : Transformations that are applied on the images to augment the dataset
  • size : Size of images / Chosen value : 224 by 224 pixels
  • valid_pct : 0.2 (i.e. 20% of the dataset will be used for validation since there aren’t specific folders for training and validation sets.

Use data.classes and you will that the dataloader considered each folder as a class as expected. The data.show_batch will help you visualize samples of your data as shown below:

Use the cnn_learner function, a built in fastai function to load a pretrained Resnet34

learn = cnn_learner(data, models.resnet34, metrics=error_rate)

Basically, the function needs the three following arguments :

  • The DataBunch
  • The specification of the model to be downloaded and trained.
  • And the metric (I chose error_rate here)

You will be running the training over 4 epochs

learn.fit_one_cycle(4)

Now that you retrained the last layer, you should unfreeze the model to retrain it as a whole:

learn.unfreeze()

In order to choose a good learning rate, you can use the built-in fastai function learn.lr_find() . This method helps you find an optimal learning rate. It uses the technique developed in the 2015 paper Cyclical Learning Rates for Training Neural Networks, where we simply keep increasing the learning rate from a very small value, until the loss stops decreasing. We can plot the learning rate across batches to see what this looks like.

It should yield a plot similar to the following if you use learn.recorder.plot()

Now, you should run the training for 2 more epochs using Learning Rate values belonging to the interval where the loss decrease is very steep : i.e. (3e-05 and 3e-04)

learn.fit_one_cycle(2, max_lr=slice(3e-5,3e-4))

The max_lr=slice(start, end), will tell your model to train the first layers with a LR of start; the last layers at a LR of end; and for the remaining layers, spread the LR across the range (start, end).

And here you go ! you should be getting better results (I got an error rate of 4%)

You can plot the confusion matrix as follows:

interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix()

Congratulations ! You have built your meme classifier ! Now you are going to deploy it !

3. Deploying the app on Render.com

In this step you will be able to deploy your meme detector to the web as I did here :

https://meme-detector.onrender.com/

Fork this github repo :

Create an account on Render.com

Upload the trained model file created with learner.export (export.pkl) to a cloud service like Google Drive or Dropbox. Copy the download link for the file.

Customize the app for your model

  1. Edit the file server.py inside the app directory and update the export_file_url variable with the URL copied above.
  2. In the same file, update the line classes = ['black', 'grizzly', 'teddys'] with the classes ['meme', 'not_meme'].
  3. You should also edit the index.html with the text you want to show for your app.
  4. In the Dockerfile, you should add the following right after the RUN pip install --upgrade -r requirements.txt
RUN pip uninstall — yes Pillow
RUN pip install ‘Pillow==6.1’

Commit and push your changes to GitHub. Make sure to keep the GitHub repo you created above current. Render integrates with your GitHub repo and automatically builds and deploys changes every time you push a change.

Deploy

  1. Create a new Web Service on Render and use the repo you created above. You will need to grant Render permission to access your repo in this step.
  2. On the deployment screen, pick a name for your service and use Docker for the Environment. The URL will be created using this service name. The service name can be changed if necessary, but the URL initially created can’t be edited.
  3. Click Save Web Service. That’s it! Your service will begin building and should be live in a few minutes at the URL displayed in your Render dashboard. You can follow its progress in the deploy logs.

Testing

Your app’s URL will look like https://service-name.onrender.com. You can also monitor the service logs as you test your app.

Conclusion

In this tutorial you learned the following skills :

  • A straightforward approach to create your own image dataset using google image.
  • Training a resnet34 and choosing the right learning rate interval.
  • Exporting the model and deploying it as a web app.

Special thanks to Jeremy and Rachel for their amazing fastai deep learning class.