The dilemma of COVID-19, masks, and Facial Recognition Systems!

Original article was published by Vaibhav Agarwal on Artificial Intelligence on Medium


With rising air pollution levels, bush fires, and more possible pandemics: the facemask is here to stay!

The dilemma of COVID-19, masks, and Facial Recognition Systems!

COVID-19 has not only caused chaos in our daily lives but in those data scientists as well! Existing solutions cannot identify people with masks, and a need for a solution is inevitable!

Turns out, COVID-19 and face masks are causing data scientists to rack their brains around the globe! Not only are they effective against airborne diseases, but research also says they’ve been blocking Facial Recognition Systems as well!

One significant consequence of it is that major Facial Recognitions Systems in place for crime prevention and security are not fulfilling their purpose. This means that if the technology does not evolve quickly, crimes may rise!

Leaked documents from the US show that Homeland Security is already about masks breaking Facial Recognition Systems! The National Crime Records Bureau in India wants its Facial Recognition Systems enhanced to include masks in it!

What is Facial Recognition?

For the uninitiated, Facial Recognition Systems are Artificial Intelligence, Computer Vision-based Deep Learning systems to detect and identify human faces. It creates facial signatures, or gives an ID to your face!

It uses mathematics to identify the relative position of your facial features and measure specific characteristics. This means calculating distances between your mouth, eyes, nose, etc. Combined, this forms your facial signature.

Facial recognition systems form a face signature from your photo! Image: pixabay/ teguhjatipras

Masks interfere with these systems, reducing the visibility of features and characteristics, thus interfering with the calculations.

The US National Institute of Standards and Technology (NIST) tested 89 algorithms to evaluate if masks impacted the performance of Facial Recognition Systems used around the globe. Results left them gobsmacked:

  • Even the most advanced models failed to detect facemasks!
  • Depending on the algorithm, error rates (total faces not detected correctly) ranged from 5% to 50%
  • Few of the models could not even identify faces if face masks covered a substantial part of the face!
NIST digitally superimposed face masks to immigration photos! Image: NIST

The problem is, no datasets of mask-wearing people to train the FRS models are available. NIST superimposed masks over the faces of real people to address this problem!

Images of real people wearing different kinds of (real!) masks would allow more details to be extracted, but this proves to be a start!

There is a confusion: Face Mask Detection Systems, like those of LeewayHertz, are different. They use the same Computer Vision-based technology to detect face masks, not faces to identify people!

LeewayHertz’s solution detects masks, but it does not identify people! Image: LeewayHertz

Use of masks will only rise

Even though the world is opening up and battling the pandemic, governments worldwide have made it mandatory for people to wear masks!

In Japan, China, and parts of India, people are wearing masks to protect against air pollution!

Political activists in Hong Kong have resorted to face masks to dodge Facial Recognition Systems.

Political Activists in Hong Kong using Face Masks to dodge Facial Recognition Systems! Image: NYTimes

If the pandemic wasn’t there, the recent bush fires might have caused people in California to wear facemasks!

As climate changes and the world develop, experts warn of rising air pollution levels, more possible pandemics, and increase in bushfires producing smoke. For a proportion of people, wearing facemasks may become a norm!

Facial Recognition Systems will need to evolve and adapt fast, meaning solutions based on visible features such as eyes, nose, eyebrows, maybe hairline, and even the face’s shape!

Technology continues to evolve, but in this case, its like a vaccine. The sooner we get one, the better!

Are there solutions to address this problem already out there? If not, can you think of a possible solution to address this?