Using Deep Learning to reconstruct fingerprints —  With Keras

Original article can be found here (source): Deep Learning on Medium

Using Deep Learning to reconstruct fingerprints — With Keras 💖

Spoiler Alert: Autoencoders are awesome

Fingerprints are an important facet of biometric security techniques — they are unique, hard to steal and every person on Earth is born with one. (Almost everyone, see: Adermatoglyphia)

They form the basis of many criminal investigations, and are also used as proof identity around the world. Unfortunately, what happens often is that fingerprints tend to be corrupted — partially erased, obscured by ink, missing, et al.

To this end, I wanted to test the plausibility of imputing missing/deformed fingerprint data from existing data. And thus began my foray into fingerprint reconstruction.

Problem Statement 🔎

Given an incomplete, or noisy sample of a fingerprint image, leverage the power of autoencoders to automatically reconstruct the complete fingerprint details.

Anatomy of a fingerprint. Source: Google Images

My first step was to look for existing work, of course. I found a few papers which used statistical methods to reconstruct minutiae points and the ridges. However, deep learning approaches to this problem seemed sparse.

But I did discover two tutorials which dealt with exactly this problem — using convolutional autoencoders: one from Datacamp and another from KDNuggets. These helped me formulate the solution architecture itself.

Armed with this knowledge, I moved onto finding a dataset.

Dataset ℹ️

A little poking about on the Internet lead me to the Sokoto Coventry Fingerprint Dataset (SOCOFing) on Kaggle. By the description on Kaggle:

This dataset consists of of 6,000 fingerprint images from 600 African subjects and contains unique attributes such as labels for gender, hand and finger name as well as synthetically altered versions with three different levels of alteration for obliteration, central rotation, and z-cut. For a complete formal description and usage policy please refer to the following paper: https://arxiv.org/abs/1807.10609

Sample of a fingerprint from the SOCOfing dataset

As part of preprocessing, we rescaled the images to be 100×100 and eliminated blank space around the fingerprint itself. We also experimented with using different filters on the image to improve the deep learning model’s understanding of the structure, but we are yet to find a suitable filter.

Convolutional Autoencoder 🔨

That’s a rather intimidating word, huh? Fret not, it’s easier than it sounds. Let’s go through this one word at a time.

Convolution

According to Wikipedia:

In mathematics (in particular, functional analysis) convolution is a mathematical operation on two functions (f and g) that produces a third function expressing how the shape of one is modified by the other.

Let’s see if we can understand it in simpler terms.

Convolution in a nutshell. Source: towardsdatascience

In 2D convolution, the process is simple. We have a matrix of data (read: pixels of an image), and we wish to extract the features in it. So we choose a smaller matrix called the kernel/filter, and slide it over the larger matrix, applying a multiplication operation at each step. The way the output of multiplication is combined into one is called “pooling”, and can be done in a variety of ways, depending on our goal.

Different aspects of this algorithm can be customised, like the kernel, the pooling, the stride, the padding, et al. For a deeper dive into this topic, see this wonderful article by Towards Data Science.

Autoencoders

An autoencoder is a neural network architecture used to efficiently learn data representation in an unsupervised manner. This architecture has many uses, such as dimensionality reduction and data denoising.

Autoencoder Schema. Source: Keras Blog

Let’s write a simple Convolutional Autoencoder in Keras:

Convolutional Autoencoder in Keras

And that’s it! This model takes in a 100x100x1 array (100px *100px, 1 color channel), and outputs an array of the same shape.

The hyperparameters for this model have not been tuned, but are the result of arbitrary choices and trial-and-error. As you can see, the model is simply a series of Conv2D and MaxPooling layers, followed by Conv2D and UpSampling2D layers. Upsampling is simply the inverse of Pooling, since it increases the matrix dimensions by performing matrix multiplications.

Results

Upon training this model on the SOCOfing dataset with the Adam optimiser and Binary Cross-entropy as the loss function of choice, we saw impressive results:

Reconstructed image (left) vs Original Image (right)

Apologies for the terrible picture quality — there is only so much that a 100×100 image can do 😭 . The autoencoder has clearly removed the distortion present in the original image with a likely approximation of what should be there.

Drawbacks

Clearly, the entire image suffers a loss of quality when put through the autoencoder. An interesting alternative would be to create another model which can identify which part of the image has been distorted, and selectively apply reconstruction. Such an approach will reduce the loss of quality dramatically. However, due to the lack of a readily available dataset, we haven’t done this yet.

Conclusion 💭

Clearly, this is a very simple approach to a very complex problem. I intend to continue working on this particular problem, while trying out different approaches. If I happen to make substantial progress, rest assured that I will update this page. Cheerio! 👋