Lessons Learned Pushing a Deep Learning Model to Production

Original article can be found here (source): Deep Learning on Medium

Lessons Learned Pushing a Deep Learning Model to Production

I tried using mybinder.org for publishing my model on the web. Here’s what worked (and didn’t).

If you, like me, are currently studying or working with ML (whether deep learning or not), are using Jupyter notebooks, and are interested in simple rapid ways to share your work via the internet, then this post is for you. Hopefully it will save you some time and make your introduction to pushing your models to production a little bit easier.

There are several things this post will not cover. There are an increasing number of tools being built to rapidly enable pushing models to production, including Render, Binder & Voila. Rather than covering all possible hosting solutions, this post will focus on an end-to-end walk through of one option: using Binder and Voila. This works best for small projects where your model .pkl file is under 50MB, and you want a free hosting service. If you need to host a larger model or get instantaneous results AWS, Google Cloud, and many other companies have hosting solutions available so check out their offerings.

Ok, let’s do this!

Step 1. Have a model ready that you want to put into production. What does “ready” mean? It means your model is in a jupyter notebook hosted on github, you have exported a .pkl file, and you have included code for simple UI elements such as a button to upload an image and a button to produce a classification. If you want to test out the pushing to production part with an existing model, you can fork the snowperson model at www.github.com/megano/snowpeopleApp.

Step 2. Ensure your file structure and sizes will play nice with Binder.

  • The model is in it’s own repository. In my case, the snowpeopleApp repo
  • The export.pkl file is *less than* 50MB. Originally I had a .pkl file about 80MB which I uploaded via github lfs and that didn’t work. Binder picked up the *reference* to the .pkl but not the actual model.

Step 3. Sign up for an account at mybinder.org.

Step 4. Paste your repository URL. In my case, it’s “https://github.com/megano/snowpeopleApp”.

Step 5. Click the “file” dropdown and change it to “URL”.

Step 6. Paste the path to the URL, inserting “/voila/render/” before your file name. My notebook is called “snowpeople_voila.ipynb”, so my URL path is “/voila/render/snowpeople_voila.ipynb”. You’ll use “/voila/render/my-nb-name.ipynb”, replacing “my-nb-name” with the name of your notebook. Troubleshooting tips, if you clicked “launch” and got a 404 message:

  • Make sure your URL includes “/voila/render/”
  • Make sure your URL does *not* have the word “master” or “blob” in it. If you’re on github, click into your .pynb file and paste that link it’s not going to work. While your link may look like “https://github.com/my-github-handle/my-repo-name/blob/master/my-notebook-name.ipyn” on github, binder needs it in a different format without the “/blob/master” part.
  • Check that the path URL/File drop down still says “URL”, and hasn’t switched back to “File” before you click “launch”.

Step 7. Time to launch! Click the “launch” button. Below is what your classifier will look like once it’s live.

Hurray! Now you can share your model with friends, family and the world. Note that, on the first run, binder can take 5 or so mins to build the app. Don’t be worried if it doesn’t work right away. When you share or open your link in a new tab, you might see something like this:

Think of this as the loading screen. After a few seconds, you should see your app again, without all the underlying notebook info.

Huzzah! You have pushed your model into production. Go and celebrate.

Want to play with the snowpeopleApp? It’s live on binder. Debugging your notebook or requirements files? Compare yours to the snowpeople repo here. Curious about what kind of data the snowpeopleApp was trained on? Stay tuned for a future post.