How to use AutoKeras to build Image Classification models using one line of code?

Original article was published by Varshita Sher on Deep Learning on Medium


DEEP LEARNING FOR BEGINNERS

How to use AutoKeras to build Image Classification models using one line of code?

No need to even know what a Conv2d, Maxpool, or Batch Normalization layer does!

How quickly can you classify people into age groups based on their hands? You are one-line-of-code away from knowing. Image Source

Disclaimer: I was tempted to write this article mainly because I was unable to find many tutorials that demonstrate how to use AutoKeras with self-collected custom datasets (i.e. datasets other than popular deep learning datasets like MNIST, ImageNet, or CIFAR-10). Additionally, with the latest version of TF rolled out, many functions (used in existing tutorials) are now obsolete and require an update.

TL;DR

1 line of code for people in a hurry:

ImageClassifier(max_trials = 200).fit(x = X, y = y, epochs = 3, validation_split = 0.2)

Introduction

In our previous articles on Deep Learning for Beginners, we learned how to build an image classification model using Pytorch and Tensorflow. For both models, I intentionally avoided going into details of the hyperparameter optimization process or tinkering with the complexity of the network architecture, both of which are useful for improving the accuracy of models.

Hyperparameter optimization (HO) is the technique of finding the optimal values of different model hyperparameters such as learning rate, etc.

You might have come across hyperparameter tuning in machine learning projects using Grid Search cross-validation. The purpose of doing so is to find the “best” model amongst all competing models. With respect to deep learning, however, the process of finding the best model can be quite challenging especially because of the somewhat cryptic black-box architecture of neural networks.

This is because there are quite a few hyperparameters that can be tuned in a neural network, for instance, number of layers, no. of neurons, activation function, number of filters, stride size, etc that might impact the performance of the network. Although it may seem like a lot to simultaneously find optimal values for each of these, we have an easier option at our disposal.

Say hello to AutoKeras.

Autokeras

AutoKeras: The holy grail of applied deep learning for creating best-of-best neural network architectures, with very little domain/technical expertise.

AutoKeras was developed by DATA Lab at Texas A&M University and its primary goal is to enable deep-learning non-experts to quickly whip up a model with minimum domain knowledge. Let’s see how…

Dataset

We will be working with the Chest X-Ray Images Kaggle dataset. Once you download and unzip it, you will see the chest_xray folder that contains images organized neatly into train, valid, and test subfolders. Each of these subfolders contains two sub-sub folders, one for normal chest Xrays and another for Xrays indicatingpneumonia. We will only be working on the ‘train’ and ‘test’ subfolder.

Note: Make sure the chest_xray folder is sitting in the same directory as your Python Jupyter notebook.

Helper functions

We will be creating a python script called helper.py to store the helper functions we will be using. For now, there is just one — img_preprocess.

def img_preprocess(img):
"""
Opens the image and does some preprocessing
such as converting to RGB, resize and converting to array
"""
img = Image.open(img)
img = img.convert('RGB')
img = img.resize((256,256))
img = asarray(img)/255

The function takes as input an image and does three things to it. First, we open the image using Image module from PIL library and convert it to an RGB format. This is quite useful because even though the images might look like they are black-and-white in color, there could be a few colored ones that might have crept into your dataset and are indistinguishable to the naked eyes. Secondly, we are going to resize all the images to size 256 x 256. This is to bring uniformity to the dataset. Finally, we are going to convert the image to a NumPy array using asarray function within numpy library. This is because the input to the train function in Autokeras expects a NumPy array.

Note: In earlier versions of TF, we could simply use the load_image_dataset method within the autokeras.image_supervised module to convert the raw image into a NumPy format. Because the newer version does not support it anymore, hence we had to make use of the asarray method.

Let’s get training…

We will be working on a new python script called AutoKeras_ImageClassification.py in Jupyter Notebook.

Importing libraries

We will begin by importing the necessary libraries and the helper function we created above.

import glob
import pandas as pd
import numpy as np
from sklearn.utils import shuffle
import cv2
from PIL import Image
from numpy import asarray
import autokeras as ak
from tensorflow import keras
from helper import img_preprocess

Note: The two files helper.py and AutoKeras_ImageClassification.py must be in the same directory.

Creating the train and test sets

As I mentioned before, we need to input NumPy arrays as the x parameter into the Autokeras’ train method. To do so, we can simply keep on picking each image present in the ‘pneumonia’ and ‘normal’ folder in chest_xray/traindirectory and send it to the img_preprocess method to create the input for the train method.

However, there is one big problem with that!

If you read all the images from normal xrays followed by all images in the pneumonia xrays folder, you have unintentionally administered some order effect into your model. That is, the model thinks that the order in which data is fed has an impact on the output, which is clearly wrong and can lead to the classic overfitting problem.

Moreover, we will be creating a validation set from the train set during training (as you will see later on) and if we had all the normal chest x-ray images at the end of the training dataset, it could mean your validation set has misrepresentation of classes. So you MUST shuffle data yourself rather than relying on Keras to do it for you!

A workaround for the aforementioned problem is: create a data frame with two columns: filename and label (0: normal and 1: pneumonia) and shuffle this data frame.

def _create_df(path, directory):
"""
Creates a df (with shuffled entries)
with two columns: file name and label (0: normal or 1: pneumonia)
"""
files1 = glob.glob (path + directory + "/NORMAL/*.jpeg")
files2 = glob.glob (path + directory + "/PNEUMONIA/*.jpeg")
# creating the train set
df_n = pd.DataFrame()
df_p = pd.DataFrame()
df_n['name'] = [x for x in files1]
df_n['outcome'] = 0.
df_p['name'] = [x for x in files2]
df_p['outcome'] = 1.
df = pd.concat([df_n, df_p], axis = 0, ignore_index = True)
df = shuffle(df) # shuffle the dataset

return df

We then read each file from this data frame and use the helper function to create the numpy representation of the image.

def create_x_and_y(self):
X = np.array([img_preprocess(p) for p in self.train_df.name.values])
y = train_df.outcome.values

return X, y

This is how the complete code, contained within a class calledAutoImageClassification, looks like:

class AutoImageClassification:
def __init__(self, path):
self.train_df = self._create_df(path, "train")
self.test_df = self._create_df(path, "test")

@staticmethod
def _create_df(path, directory):
"""
Creates a df (with shuffled entries)
with two columns: file name and label (0: normal or 1: pneumonia)
"""
files1 = glob.glob (path + directory + "/NORMAL/*.jpeg")
files2 = glob.glob (path + directory + "/PNEUMONIA/*.jpeg")
# creating the train set
df_n = pd.DataFrame()
df_p = pd.DataFrame()
df_n['name'] = [x for x in files1]
df_n['outcome'] = 0.
df_p['name'] = [x for x in files2]
df_p['outcome'] = 1.
df = pd.concat([df_n, df_p], axis = 0, ignore_index = True)
df = shuffle(df) # shuffle the dataset

return df

def create_x_and_y(self):
X = np.array([img_preprocess(p) for p in self.train_df.name.values])
y = self.train_df.outcome.values

return X, y

Finally, to train our neural network, we will create an object of class AutoImageClassification, call the create_x_and_y method to create inputs x and y for the train function. To begin the training process, we simply need one line of code that uses the fit method.

aic = AutoImageClassification(path = "chest_xray/")X, y = aic.create_x_and_y()# autokeras
model = ak.ImageClassifier(
max_trials = 200,
loss = 'binary_crossentropy',
metrics = 'accuracy' )
model.fit(x = X, y = y, epochs = 3, validation_split = 0.2)

Note: The training process will take a really long time so grab a cup of coffee or better yet, make use of the free GPUs available on GoogleColab Notebook to fasten the process.

ImageClassifier is the Autokeras image classification class. To initialize, the max_trials parameter is set to 200, meaning 200 different Keras models will be tried (default value is 100). The accuracy metric will be used to determine how good the model is and at each iteration, the binary_crossentropy loss will be monitored.

The fit method is the place where the magic happens. It will search for the best set of hyperparameters for our model and in turn generate the best model based on our data. It takes as input, predictors x and outcomes y. We also specify epochs as 3 meaning the model is going to train on all of the data 3 times. Finally, we are setting aside a small proportion (20%) of the training data during each epoch to be used as a validation set. This is going to help us ensure our model is not overfitting and is able to generalize well on unseen data.

Once the training process starts, the first few lines of output should look something like this:

Evaluating the model

We will be using the test set to make predictions. But first, we must preprocess the images present in chest_xray/testdirectory in the same manner as the trainset images.

x_test    = np.array([img_preprocess(p) for p in aic.test_df.name.values])
y_test = aic.test_df.outcome.values

To make predictions, simply use the predict method:

# test data
y_pred = model.predict(x_test)
y_pred = [round(y[0],0) for y in y_pred]
print("accuracy = ", accuracy_score(y_test, y_pred))
print("recall = ", recall_score(y_test, y_pred))
print("precision = ", precision_score(y_test, y_pred))
print("f1 score = ", f1_score(y_test, y_pred))
Evaluation metrics on test data

Saving the model for future use

In order to export the trained model:

# Export as a Keras Model. 
model = model.export_model()
model.save('models/xray_classification.h1')

To load it again:

from tensorflow.keras.models import load_model
new_model = load_model('models/xray_classification.h1')