I tried replicating the recommendation engines used by e-commerce platforms, here’s how it went….

Original article was published on Deep Learning on Medium


I tried replicating the recommendation engines used by e-commerce platforms, here’s how it went….

As a consumer for various e-commerce websites, I was always intrigued when the platforms started recommending items that were similar to previous purchases, searching history or even just items that looked alike. With quarantine having left me with extreme boredom, I decided to dive further into the world of Recommendation engines and boy was I disappointed. Turns out, the algorithms are not very difficult to understand. Therefore, I decided to create a recommendation system for the fun of it.

The Outline

Data Collection

Creating a Transfer Learning model and Saving it

Getting recommendations

Creating a User Interface and embedding the model

Deployment

Spoiler : I did make a static naive model which works for some static images. Click here to get recommendations!

It is evident that there a ton of recommendation engines only but most of them only cater to the digital part of the spectrum. Movie recommendations are the most implemented type of recommendation systems. So, trying to figure out how to create a model was quite a task. In the end, I decided a transfer learning model can be used for extracting features and as seen by many tutorials only cosine similarity can be used for getting recommendations.

Note: Code snippets are added, but for the entire source code, Click here.

Data Collection

Data collection is easy. Write some python code and you will have your data ready. I decided to scrape my own data but any dataset having a collection of product images is fine really! Kaggle has many such datasets.

Here’s some code for data scraping- I first scraped the image links from the website and then downloaded the images.

Transfer Learning

Transfer learning is simply the manipulation and implementation of a model created to solve one problem that is used to solve a different but related problem. Basically, using a model used to classify cats and dogs to classify burgers and pizzas. There are many models available viz. InceptionV3, VGG16,ResNet models and all serve the same purpose but have different architectures. I chose the VGG16 model for feature extraction.

Feature extraction is the process of evaluating images and identifying manageable groups for processing from an initial set of raw variables.

After feature extraction, Cosine similarity was used to identify similar images.Cosine similarities is calculated using the below formula.

Cosine similarity

This basically creates a utility matrix which can be used further to index the images and get the most similar ones!

Below is some transfer learning code :

# load the model
vgg_model = vgg16.VGG16(weights='imagenet')

# remove the last layers in order to get features instead of predictions
feat_extractor = Model(inputs=vgg_model.input, outputs=vgg_model.get_layer("fc2").output)

# print the layers of the CNN
feat_extractor.summary()

Click here for the entire notebook!

Getting recommendations

Getting recommendations was easy after feature extraction. Just indexing the images and getting similar items was the goal.

The below function was used:

def retrieve_most_similar_products(given_img):

print("-----------------------------------------------------------------------")
print("original product:")

original = load_img(given_img, target_size=(imgs_model_width, imgs_model_height))
plt.imshow(original)
plt.show()

print("-----------------------------------------------------------------------")
print("most similar products:")

closest_imgs = cos_similarities_df[given_img].sort_values(ascending=False)[1:nb_closest_images+1].index
closest_imgs_scores = cos_similarities_df[given_img].sort_values(ascending=False)[1:nb_closest_images+1]

Creating a User Interface

There are many options for creating interfaces, I chose Plotly-Dash simply because it is entirely in Python with a little CSS(Bootstrap). The dash tutorials on the offical website are pretty easy to understand and implement. I figured out how to add callbacks and inputs, outputs and there I had a working model. Me being a barely creative person, it took me a couple of hours to figure the color scheme out.

This is how the model looks:

Recommendations

Deployment

Dash applications are usually deployed on Heroku. Now, I tried doing that but ran into a bunch of errors and warnings. Turns out, heroku only allows upto 500MB of slug size(prototype size) and just the size of my model was about 528MBs. So, the entirety of the application was about 800 megabytes. Therefore, I had to generate recommendations for all images on the interface and statically store them and call them everytime the user requested for one. This meant that the model was not directly used for getting predictions, but oh well, atleast the application was fast and doing the job. But, the model can be used directly on your local machine(I call this the Dynamic User Interface).

Here is the deployed model: Product recommendations👢

For the entire source code:

Thanks for reading!