Masked Face Recognition Application

Original article was published on Deep Learning on Medium

Masked Face Recognition Application

Facial recognition is the process of identifying or verifying the identity of a person using their face. It captures, analyzes, and compares patterns based on the person’s facial details. The facial recognition market grows very fast in the last few years. That’s because facial recognition has all kinds of commercial applications. Applications like criminal identification, attendance systems, face-unlock systems. But with spreading of COVID-19, millions don masks across the world. This affects the accuracy of the face identification system. The existing face recognition solutions are no longer reliable when wearing a mask and removing masks for passing authentication will increase the risk of virus infection. Suddenly, the employees can’t register themself in the attendance system since many companies suspend fingerprint devices and Mark attendance through the face is not working well. To this end, this work proposes a method that could improve the existing face recognition approaches that heavily rely on all facial feature points, so that identity verification can still be performed reliably in the case of incompletely exposed faces.

In order to handle masked face recognition, I propose some tasks we have to follow :

1. Data Processing

All faces in images and their landmarks will be detected by Multi-task Cascaded Convolutional Networks (MTCNN) algorithms [1]. We will use two eyes as landmarks for similarity transformation. When the detection fails, we simply discard the image. Dataset will be included web-collected training data, Real-world Masked Face Recognition Dataset (RMFRD) [2]. After removing the images with identities appearing in testing datasets, it roughly goes to 5,000 images of 525 unique persons.

2. Face Segmentation

We will use Deep face segmentation technique [3] which applies a fully convolutional network (FCN) to segment the visible parts of faces from their context and occlusion. All intersecting segmented regions further proceed to the next task.

3. Face Recognition

Network Setting

The proposed deep learning structure based on convolutional neural networks (CNN ) . The higher capacity network such as ResNet dill ever better prediction accuracy compared to older architectures like VGG. Nowadays, advanced technology makes training deeper network faster and hence extract a large amount of facial data ( feature vector ) with various attributes ( locations, gender, age, pattern ). The CNN model work to map face images into 100-dimensional encodings, By using a 100-neuron fully connected layer as its last layer. I believe if we specify the full 100-dimensional encoding only for the upper part of the face, we will get more distinctive details about a person’s face. These details, such as distance between the eyes or texture, the color of skin, and eyebrow shape are then converted into a mathematical representation and compared to data on others.

Training Data

The faces are cropped to 96 × 96 RGB images. The images are horizontally flipped for data augmentation. Each pixel in these images is normalized by subtracting 127.5 and then being divided by 128.

Loss Function

Loss functions play an important role in CNN training. The main goal of classifier to increase intra_class compactness enlarge the inter-class separation. Although Softmax Loss is the most widely used classification, it does not have this discriminative power. For that, Scientist developed Triplet Loss that use triplet approach to increase performance. Later, another more lose function achieves more accurate prediction like ArcFace loss or insight face.


Masked recognition application requires subjects to be approaching and facing up the camera. Thus high-quality frontal face images will readily acquired, so that the masked face recognition task will be no longer difficult. Even if the mask covers part of the face, the features of upper half of the face, such as eye and eyebrow, can still be used to improve the availability of the face recognition. Finally, Face recognition uses in last years increased exponentially with numerous use cases. It can be used for everything from surveillance to marketing. More than ever , Face recognition represents an important problem that should be studied with the utmost priority.

References :

[1] Joint face detection and alignment using multi-task cascaded convolutional networks , Zhang, K., Zhang, Z., Li, Z., Qiao

[2] Masked Face Recognition Dataset and Application , Zhongyuan Wang, Guangcheng Wang, Baojin Huang, Zhangyang Xiong, Qi Hong, Hao Wu, Peng Yi, Kui Jiang, Nanxi Wang, Yingjiao Pei, Heling Chen, Yu Miao, Zhibing Huang, and Jinbi Liang

[3] On Face Segmentation, Face Swapping, and Face Perception , Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, Gerard Medioni

[4] A Performance Evaluation of Loss Functions for Deep Face Recognition Yash Srivastava, Vaishnav Murali, and Shiv Ram Dubey