How to deploy your trained PyTorch model on Heroku

Source: Deep Learning on Medium


Introduction

This article assumes you have basic Python knowledge as well as some deep learning background and you know how to use pytorch for training deep learning models

You trained your pytorch deep learning model and tuned the hyperparameters and now your model is ready to be deployed. If you don’t know how to deploy your model then this article is for you stay tuned.

What is Heroku

Heroku Logo

Heroku is a platform as a service (PaaS) that enables developers to build, run, and operate applications entirely in the cloud.

Requirements for deploying your model on Heroku

In order to deploy your model on heroku these are the things that are required

  1. Git
  2. Flask
  3. HTML,CSS,JavaScript files for the web app
  4. Heroku account and some additional files

Requirement 1: Git

In case you have no idea what git or how to use it I recommend Udacity’s free course How to use git and github

Git Logo

Download Git on your local computer through this link and follow the installation steps It’s very straightforward.

Requirement 2: Flask

Flask Logo

To install flask use the command

pip install Flask

This is the folder structure in Flask application

/web-app
|--templates
|----index.html
|--static
|----css
|------style.css
|----js
|------script.js
|--model.pt
|--application.py

The templates folder contains all html files needed in your application. The static folder is used to store css files and javascript files. The model.pt file is the file used to load your Pytorch trained model and application.py is the python application contains the logic for your web application.

You don’t need to store your css and javascript files in separate css and javascript folders but it’s better structure

The structure for your application.py file

This is the basic structure for your flask application @app.route(‘/’) tells Flask which url will trigger the function render_page() and return render_template(‘index.html’) will render the html file index.html

By default all the methods are GET methods in case you want to specify a POST method which we will use for the prediction then we should add methods=['POST']

Here’s a model prediction example for an image uploaded by the user if the image uploaded is of human it returns most resembling dog breed if it is of a dog then it returns the dog breed if neither detected it returns a message indicating that it can find neither human face nor dog

As you can see there is nothing special about this piece of code just python but the thing the most important line of code is jsonify which is used to return a JSON Object in case you want to deal with this link as a RESTful API or you are sending an AJAX request

Requirement 3: HTML,CSS and JavaScript files

As mentioned in the folder structure all html files are stored in templates folder and any other files like CSS or JavaScript are stored in static folder but to use the CSS file in Flask you have to add small line of code in your index.html file

For CSS

<link rel=”stylesheet” href=””>

For JavaScript

<script src=””></script>

url_for function helps to use static folder inside the deployed server for loading CSS files and JavaScript files

Also when using AJAX in jQuery or JavaScript make sure that the url matches the one in your application.py folder for example in the application.py the url for POST method was /predict so it has to be the same in your javascript file

$.ajax({
type: ‘POST’,
url: ‘/predict’,
data: form_data,
contentType: false,
cache: false,
processData: false,
success: function(data) {
// what you want to do with the prediction returned
}
})

Requirement 4: Heroku and Additional Files

In order to deploy your Flask web application on Heroku you have to register and download and Install Heroku CLI

Then we have to install gunicorn by using this command pip install gunicorn then inside your web-app folder you will create three new files runtime.txt , Procfile and requirements.txt so our folder structure will be as follows

/web-app
|--templates
|----index.html
|--static
|----css
|------style.css
|----js
|------script.js
|--model.pt
|--application.py
|--runtime.txt
|--Procfile
|--requirements.txt

Runtime.txt

python-3.7.0

Heroku runs on Python 3.7 or later that’s why we need to provide this file

Procfile

web: gunicorn application:app --log-level debug

where application is the name of your .py file and for --log-level debug is for debugging purposes to be able to debug any errors in your heroku application

requirements.txt

In this file you should add the dependencies of your project in the file requirements.txt but that will be tedious another approach is using pip freeze

pip freeze > requirements.txt

But this may introduce unnecessary depenedencies and it may also cause errors when you deploy your app to Heroku better approach is to create requirements.txt file which has all the dependencies of your project using pipreqs install it using pip install pipreqs then

cd ..
pipreqs web-app

Then in your requirements.txt file replace torch== X.Y.Z where X,Y,Z are version numbers of PyTorch with https://download.pytorch.org/whl/cpu/torch-1.0.1.post2-cp37-cp37m-linux_x86_64.whl

Also add gunicorn == x.y.z where x,y,z are version number of gunicorn and to get the version number of gunicorn use these two python lines of code

import gunicorn
print gunicorn.__version__

Now you’re ready to deploy your model to Heroku

Steps Needed to deploy your app to Heroku

Login to your heroku account either through the website or heroku cli using the command

heroku login

Then create a heroku web app either through your account or through cli using the command

heroku create

Then deploy your application to heroku using these set of commands

git init
git add .
git commit -m 'initial commit'
git push heroku master

then open your application using either by copying the link or using the command

heroku open

Notes for deploying your app to Heroku

  • Train smaller models like resnet50 or densenet121 as Heroku free tier size is 512MB so large applications may cause memory error and failure of your application
  • Sometimes you will encounter errors regarding some missing dependencies add any missing dependencies to the file requirements.txt
  • Beware of memory issues and free any unused variables in your application

Conclusion

Here’s my github project which is great example for deploying your trained model on Heroku. Also Flask documentation is another great way to learn it’s very easy to learn and master