One Shot Learning using Keras

Original article was published on Deep Learning on Medium

One Shot Learning using Keras

Image Source: https://github.com/brendenlake/omniglot

Abstract

For a neural network to learn features from images in order to classify them we need data, lots of data. It is difficult for a model to learn from very few samples per class.

MNIST dataset has nearly 60000 training images for numbers 0–9 (10 classes).

We will implement One shot learning to build a model which will correctly make predictions given only a single example of each new class.

Background

As humans , when we are presented with new object , we quickly pickup patterns , shape and other features. When we are presented with same kind of object in future , we quickly recognize that object. We all can relate to such instances. Suppose you saw a blender for the first time in your friends kitchen. Later when you see it at any store , you quickly recognize that its a blender (regardless of brand), not any other equipment.

Despite the fact that the person has only seen this object once in their life, he still differentiate that object from others. This is known as one shot leaning.

prediction using Images (left) and speech (right)

To quote from this paper –http://www.cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf

One particularly interesting task is classification under the restriction that we may only observe a single example of each possible class before making a prediction about a test instance. This is called one-shot learning

Image Classification vs One shot Learning

To understand the difference , consider a Face Recognition System designed for a company with 10 employees. Now, for this we can use traditional CNN approach to build a model, by collecting enough images for each person(class) and fit a classification model, that will precisely predict the person given an image. When implementing face verification , where we match the face of the person with the one in database, we need to make sure that the both images are of same person.

This approach will have few issues:

  1. If a new employee joins the firm, then we need to go through the same process again , and re-train the model. This is same when an employee leaves the company.
  2. As the number of employees increase, the dataset will increase and it will be difficult to fit the model.
  3. We may not get sufficient samples for each class every time. Ideally a new employee may only give 1-2 images while joining.

One shot learning tackles this issue.

Given pair of images, the model predicts the degree of similarity between them.

Rather than predicting the class of image , we will try to predict the degree of similarity between pairs of image. This is done by representing images in N-Dimensional embedding vector which the model will generate. When these embedding vectors are represented in 2-D space , the distance between embedding of similar images will be less . Hence, the model predicts whether the images are similar or not.

In Face Recognition this is done in 128-D embedding vector

For further information on face recognition using one-shot , watch Andrew Ng’s Video.

Few applications of one shot learning:

  1. Face Recognition , learn more here.
  2. Drug Discovery, learn more here.

Problem Statement

Our model is given a tiny labelled training set S, which has N examples, each vectors of the same dimension with a distinct label y.

Content source: https://github.com/sorenbouma/keras-oneshot

It is also given x_test, the test example it has to classify. Since exactly one example in the support set has the right class, the aim is to correctly predict which y ∈ S is the same as x_test’s label.

Image source: https://github.com/sorenbouma/keras-oneshot

But the problem becomes challenging when N increases. Now we have to compare our test image with N different images and look for the highest probability for the correct class.

Dataset

I have used Omniglot dataset by Brenden Lake.