Facial recognition login system using Deep learning + ReactJS

Original article was published on Deep Learning on Medium

You must have seen a facial recognition login system in apps or websites. If you are an app developer or a website developer you may want to add facial recognition as a login phase in your system. Here, we are going to build a very easy-to-adapt login system from scratch that you can integrate into any app or system for login aspects.

We will build this system in 2 conspicuous steps. These steps include creating a Flask server that contains our deep learning model and creating a ReactJS app for login. ReactJS application could communicate with our deep learning model with the help of the Flask server. It can send the image of the person and the model can return the true identity of the person in the image. These all will be clear once we start coding. Let’s get to the coding part and create our login system. Take a look at the below figure to clear any misgivings about our system to be built.

The simple flow of the system

Creating a deep learning model

Facial Recognition and Facial Verification are two different things. Facial recognition is concerned with identifying who is the person in the image by detecting the face while facial verification determines if the given two images are of the same person or not. Here, to build our login system we will require a model that can detect faces from the images and convert the facial details into a 128-dimensional vector which we can use later for facial recognition or verification purposes.

We can use the FaceNet model provided by Hiroki Taniai so that we do not have to build one from scratch and need to worry about the hyperparameters. This FaceNet model has been trained on the MS-Celeb-1M dataset and expects a color image of size 160×160 pixels. The model can be downloaded from here: Keras FaceNet Pre-Trained Model.

First, let’s import important modules for our Flask server that contains our deep learning model.

# Import all the necessary files!
import os
import tensorflow as tf
from tensorflow.keras import layers
from tensorflow.keras import Model
from tensorflow.python.keras.backend import set_session
from flask import Flask, request
from flask_cors import CORS
import cv2
import json
import numpy as np
import base64
from datetime import datetime

Some important steps for our API to work. You can learn more about why we use sessions in Flask here.

graph = tf.get_default_graph()app = Flask(__name__)
CORS(app)
sess = tf.Session()
set_session(sess)

Finally, we will load our model.

model = tf.keras.models.load_model('facenet_keras.h5')

Let’s create a function that converts the given image into a 128-dimensional vector. This function takes the image path and our model as parameters and outputs the vector that contains information of the image.

def img_to_encoding(path, model):
img1 = cv2.imread(path, 1)
img = img1[...,::-1]
dim = (160, 160)
# resize image
if(img.shape != (160, 160, 3)):
img = cv2.resize(img, dim, interpolation = cv2.INTER_AREA)
x_train = np.array([img])
embedding = model.predict(x_train)
return embedding

Let’s create a sample database that stores the identity of the people. You can store some identities beforehand if you want. You can give an image of a person as an argument to the img_to_encoding method along with the model and you will get the 128-dimensional vector which you can store as a value to the database dictionary with the person’s name being the key.

database = {}
database["Klaus"] = img_to_encoding("images/klaus.jpg", model)
database["Levi"] = img_to_encoding("images/captain_levi.jpg", model)
database["Eren"] = img_to_encoding("images/eren.jpg", model)
database["Armin"] = img_to_encoding("images/armin.jpg", model)

Let’s create a function that will return the identity of the person in the image using the database. It will compute the distance as shown above for all the entries in the database and the new image and find the minimum distance. If the minimum distance is larger than the threshold it will show that the person is not registered otherwise it will return the identity with the minimum distance.

def who_is_it(image_path, database, model):
encoding = img_to_encoding(image_path, model)
min_dist = 1000
#Looping over the names and encodings in the database.
for (name, db_enc) in database.items():
dist = np.linalg.norm(encoding-db_enc)
if dist < min_dist:
min_dist = dist
identity = name
if min_dist > 5:
print("Not in the database.")
else:
print ("it's " + str(identity) + ", the distance is " + str(min_dist))
return min_dist, identity

Finally, we want our Flask server to do two tasks: 1) Register the user to the database dictionary if not registered 2) Identify the identity of the user from an image. We will create routes for our API. First, we will create a route that identifies the user from an image and returns the identity. This route will receive base64 data of the image from React app captured by the user at a time and it will return the identity of the user if registered or it will send a hint that the user is not registered so that our React app knows whether to log in. Note: Go here if you want to know the reason behind using the TensorFlow sessions.

app.route('/verify', methods=['POST'])
def verify():
img_data = request.get_json()['image64']
img_name = str(int(datetime.timestamp(datetime.now())))
with open('images/'+img_name+'.jpg', "wb") as fh:
fh.write(base64.b64decode(img_data[22:]))
path = 'images/'+img_name+'.jpg'
global sess
global graph
with graph.as_default():
set_session(sess)
min_dist, identity = who_is_it(path, database, model)
os.remove(path)
if min_dist > 5:
return json.dumps({"identity": 0})
return json.dumps({"identity": str(identity)})

Above we created a temporary image file for our purposes and after checking for the identity of the user we removed it as it is not useful to us. We are naming this file with the help of the timestamp.

Time to create our second route that will register the new person into the database. It will receive the user’s name and the base64 data of the user’s image from the React app. It will compute the 128-dimensional vector for the image and store it into the database dictionary. If registered successfully it will send the status code of 200 otherwise it will send 500 status code (internal error).

@app.route('/register', methods=['POST'])
def register():
try:
username = request.get_json()['username']
img_data = request.get_json()['image64']
with open('images/'+username+'.jpg', "wb") as fh:
fh.write(base64.b64decode(img_data[22:]))
path = 'images/'+username+'.jpg'
global sess
global graph
with graph.as_default():
set_session(sess)
database[username] = img_to_encoding(path, model)
return json.dumps({"status": 200})
except:
return json.dumps({"status": 500})

That is all for the deep learning part of the system. There are chances that you are still unclear about how these all will fit into the system we are building. Let’s take a look at the flow of the system after creating our Flask server.