Source: Deep Learning on Medium
Hi, I recently worked on a conversational UI chatbot for a wedding card website, When we analyzed the wedding cards customer care support most of the user bring some sample of the wedding cards image and they ask the customer executive person to show similar or same wedding cards, but for customer executive person it is a tedious job to find some similar wedding card quickly due to that we decided why can we give this feature to the Chatbot So that chatbot can work on the sample card and give a similar image or same card within a fraction of second. So I started working on the image similarity.
At first, I gathered some image from the google image search and also some website using the scrapy tool and I started training the image with single autoencoder to get the latent representation of each image and using the latent representation we trained the KNN to cluster the latent represented image. So if you send a new image to encoder part it will give latent representation then it is sent to the already trained KNN and KNN will give the nearest neighbor image. This technique I got referenced from the below link
In this article, we will explain how autoencoders can be used for finding similar images in an unlabeled image dataset…towardsdatascience.com
This link is awesome so take look out at it.
But, I didn’t get the best result using a single autoencoder.
So I started analyzing the wedding card type and pattern.
after the analyzation, I got some intuitive about the wedding card
At first, I classified the image data into 8groups.
Hindu wedding card
Muslim wedding card
Christian wedding card
Laser cutting wedding card
Floral wedding card
3d popup wedding card
Scroll wedding card
Box wedding card
I architected a classification model to classify this 8 label, then I architected 8 different autoencoders for this 8 label after I trained 8 KNN model to cluster the given latent represented data.
For example, If I sent a laser cutting wedding card image to the classification model it will classify as laser cutting cards then, it will choose pre-trained encoder part and send the image to the laser cutting encoder model and get the latent represented data after it is sent to the KNN it will give the similar image of laser cutting card.
So I downloaded the image for each label around 100 images, then I trained the 100 images to the autoencoder, for example, I trained 100 box wedding card image to one autoencoder. So it has a latent representation of all the 100 box wedding card, If I send a box wedding card which is in red color and small box then KNN will give first 10 similar images which also red color and small box design.
Here I used Unet as an autoencoder , but In above link they used a normal convolution autoencoder.
Contribute to vigneshgig/similiarimage development by creating an account on GitHub.github.com
In above GitHub link, you can find dataset creating notebook and UNET autoencoder notebook file but I haven’t included the xception classification code.
Note: This Xception code is not for this problem, but it is easy to modify or you can get many tutorials regarding xception classification problem.
from keras.applications.xception import Xception, preprocess_input
from keras.preprocessing import image
from keras.models import Model
from keras.layers import Input
from keras.layers import Dense, GlobalAveragePooling2D
from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
from keras.layers import Flatten
from keras.layers.convolutional import Conv2D
from keras.layers.convolutional import MaxPooling2D
from keras.layers import BatchNormalization
import numpy as np
import matplotlib.pyplot as plt
hdf5_path = '/content/drive/My Drive/monogramdataset.hdf5'
hdf5_pathtest = '/content/drive/My Drive/monogramdatasetvalidation.hdf5'
hdf5_file = h5py.File(hdf5_path, "r")
hdf5_filetest = h5py.File(hdf5_pathtest, "r")
# reshape to be [samples][pixels][width][height]
X_train = hdf5_file.get('train_img').value
y_train = hdf5_file.get('train_labels').value
X_val = hdf5_filetest.get('val_img').value
y_val = hdf5_filetest.get('val_labels').value
X_test = hdf5_filetest.get('test_img').value
y_test = hdf5_filetest.get('test_labels').value
# X_train = X_train.reshape(X_train.shape,X_train.shape,X_train.shape)
# X_test = X_test.reshape(X_test.shape,X_test.shape,X_test.shape)
X_train = X_train.reshape(X_train.shape, 299, 299,3).astype('float32')
X_test = X_test.reshape(X_test.shape, 299, 299,3).astype('float32')
X_val = X_val.reshape(X_val.shape, 299, 299,3).astype('float32')
# X_train = X_train.reshape(X_train.shape, 299, 299,1).astype('float32')
# X_test = X_test.reshape(X_test.shape, 299, 299,1).astype('float32')
# X_val = X_val.reshape(X_val.shape, 299, 299,1).astype('float32')
# create base pre-trained model
from keras.applications.xception import Xception
# from keras.applications.inception_resnet_v2 import InceptionResNetV2
base_model = Xception(weights='imagenet', include_top=False)
#add a global spatial average pooling layer
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(200,activation='elu')(x)
x = Dropout(0.4)(x)
x = Dense(170,activation='elu')(x)
predictions = Dense(157,activation='hard_sigmoid')(x)
model = Model(inputs=base_model.input, outputs=predictions)
# # we use SGD with a low learning rate
from keras.optimizers import Nadam
from keras.optimizers import adam
# model.compile(optimizer='adam', loss='binary_crossentropy',metrics=['accuracy'])
# # we train our model again (this time fine-tuning the top 2 inception blocks
# # alongside the top Dense layers
model.fit(X_train, y_train, validation_data=(X_val, y_val),shuffle="batch" ,epochs=10, batch_size=10,verbose=1)
# Final evaluation of the model
scores = model.evaluate(X_test, y_test, verbose=0,batch_size=10)
print("Baseline Error: %.2f%%" % (100-scores*100))
# # serialize model to JSON
model_json = model.to_json()
with open("/content/drive/My Drive/monogrammodelxceptioneludrop299.json", "w") as json_file:
# serialize weights to HDF5
print("Saved model to disk")
and also I have given a link for the similarimage folder which contains the pre-trained model and dataset make use of it.
Dropbox is a free service that lets you bring your photos, docs, and videos anywhere and share them easily. Never email…www.dropbox.com
Thanks for reading my blogs………..