Original article was published by Manali Seth on Deep Learning on Medium
Face Mask Detector with OpenCV, Keras/Tensorflow and Deep Learning
The ongoing novel coronavirus pandemic situation is known to each and every individual. Almost every country has been affected by the devastating Coronavirus(COVID-19) disease. The pandemic threatens to reverse hard-won gains made in global health and human capital over the past decade. The mandated confinement and social distancing measures in force during an extended period of time, will make it beneficial to flatten the curve of transmission. Artificial intelligence could play an important part in the post-COVID recovery, helping to boost productivity and foster a new generation of innovative companies. While this situation happens to worsen day by day, it is very much essential for everyone to follow some rules in order to remain safe than to get badly affected with its consequences.
The frontline warriors are working hard to save many lives but it is more of an individual’s responsibility to fight one’s own battle and not compromise their health. Few ways suggested for it are: Washing hands regularly, maintaining social distance, wear face mask regularly and staying quarantined, if unwell
Here, I have tried to design a custom deep learning model of Face Mask Detector using OpenCV, Keras/Tensorflow libraries which detects if an individual is wearing a face mask or not and alerting for the same.
TWO PHASE FLOW FOR FACE MASK DETECTOR
The dataset consists of 1,376 images belonging to two classes:
- with_mask: 690 images
- without_mask : 686 images
We perform fine-tuning on the MobileNet V2 architecture, a highly efficient architecture which can be applied to embedded devices with limited computational capacity. Deploying our face mask detector to embedded devices will help to reduce the cost of manufacturing such face mask detection systems, thus we choose to use this architecture.
Firstly, we import all the necessary libraries required for implementation some of which are :
- tensorflow.keras for Data augmentation, Loading the MobilNetV2 classifier, Building a new fully-connected (FC) head, Pre-processing, Loading image data, sklearn, imutils, matplotlib, numpy etc.
- sklearn for binarizing class labels, segmenting our dataset, and printing a classification report.
- imutils paths implementation to find and list images in our dataset.
- matplotlib to plot our training curves.
Then, we construct the argument parser and parse command line arguments required while running our code. We then specify hyperparameter constants including initial learning rate, number of training epochs, and batch size.
Further, we are ready to load and preprocess our training data and then prepare it for data augmentation which includes:
- Grabbing the list of images in our dataset directory, then initializing the list of data (i.e., images) and class images
- Preprocessing images
- Converting training data into Numpy array format
- Encode labels (One-hot encoding)
- Partition of dataset into training(80%) and testing set(20%) using scikit-learn library
Preparing MobileNetV2 for fine-tuning:
- Load MobileNet with pre-trained ImageNet weights, leaving off head of network
- Construct a new FC head, and append it to the base in place of the old head
- Freeze the base layers of the network. The weights of these base layers will not be updated during the process of backpropagation, whereas the head layer weights will be tuned.
Compile and train face mask detector network:
- Compiling model with Adam Optimizer, a learning rate decay schedule and binary cross entropy (as there are only 2 classes)
- Training the head of the network using model.fit method and its required arguments
Evaluating the resulting model on the test set by making predictions on the test set, grabbing the highest probability class label indices. Then, we print a classification report for inspection. We then serialize our face mask classification model to disk. Also, we plot our accuracy and loss curves.
Here, we can observe that we almost have 99% accuracy on our test set.
Implementing our COVID-19 face mask detector for images with OpenCV
TensorFlow/Keras imports to (1) load our MaskNet model and (2) pre-process the input image.
OpenCV is required for display and image manipulations.
Command line arguments include:
- — image: The path to the input image containing faces for inference
- — face: The path to the face detector model directory (we need to localize faces prior to classifying them)
- — model: The path to the face mask detector model that we trained earlier in this tutorial
- — confidence: An optional probability threshold can be set to override 50% to filter weak face detections
Then we load both our face detector and face mask classifier models. Next step is to load and preprocess an input image. Pre-processing is handled by OpenCV’s blobFromImage function. Then we loop over our detections and extract the confidence to measure against the confidence threshold. We then compute bounding box value for a particular face and ensure that the box falls within the boundaries of the image. Next, we will run the face ROI (extracted through Numpy Slicing) through our MaskNet model.
After that, we determine the class label based on probabilities returned by the mask detector model and assign an associated color for the annotation. The color here will be “green” for with_mask and “red” for without_mask. We then draw the label text (including class and probability), as well as a bounding box rectangle for the face, using OpenCV drawing functions. Once all detections have been processed, the output will be displayed.
Thus, we see that, the face mask detector model has correctly labelled ‘Mask’ and ‘No Mask’ in the above sample images.
Entire source code for reference is provided in my Github Repository Link https://github.com/ManaliSeth/Face-Mask-Detection
Hope this entire document be useful for all working upon concepts of OpenCV, Deep Learning and Face Mask Detection.