AutoML with AutoKeras

Source: Deep Learning on Medium

Neural networks to design neural networks

Photo by Daniele Levis Pelusi on Unsplash

Applications of deep learning in real world scenarios have increased, yet it requires an expert to build the model and tune it to improve the accuracy. Various stages in machine learning includes

  1. Data preparation
  2. Feature engineering
  3. Architectural search
  4. Hyperparameter tuning
  5. Model selection

Primary objective of a deep learning expert is to build an effective neural architecture with suitable hyper parameters. It requires several combinations to be tested before arriving at an effective model which is time consuming as well as expensive.

AutoML facilitates automation of the above machine learning tasks to build models in a highly scalable and efficient way so that it can be used widely by non experts as well. The two major pipeline steps automated are the neural network generation and tuning the hyper parameters.

Google published a paper on Neural Architecture Search with Reinforcement Learning where they used RNN to generate the model descriptors of neural network and then trained the RNN with reinforcement learning to maximise the accuracy of the generated networks. Improvements were made thereafter with evolutionary algorithms.

There are number of open source automated machine learning frameworks that includes auto-sklearn, autokeras,, MLBox, TPOT and TransmogrifAI.

Let us implement an image classifier to classify elephant and boar images with AutoKeras. AutoKeras is an AutoML library that employs Neural Architecture Search (NAS) with Bayesian Optimisation.


Install the following libraries

pip install tensorflow
pip install keras
pip install autokeras

We will need to collect images of both the animals and then resize it to a dimension of (128 * 128 * 3). Then we can split the resized images into train and test datasets.

CSV file for test and train datasets should be created with image and label information which are then used to load the train and test datasets

from autokeras.image.image_supervised import ImageClassifier, load_image_datasettrain_data_path = "path/to/train/images"
train_csv_path = "path/to/train.csv"
test_data_path = "path/to/test/images"
test_csv_path = "path/to/test.csv"
x_train, y_train = load_image_dataset(csv_file_path=train_csv_path,
x_test, y_test = load_image_dataset(csv_file_path=test_csv_path,

Once the train and test datasets are loaded we will use the ImageClassifier to start the Architecture search. We can specify the path to save the generated models as well the time_limit to perform search and build models. Parameters such as learning rate, hidden layers, activation are automatically handled by AutoKeras.

model = ImageClassifier(path='path/to/save/models/', verbose=True), y_train, time_limit=60 * 60)

AutoKeras starts with a simple model and then continues to build models until the specified time_limit. Here the limit is set to 1 hour.

After an hour various different models are generated and best model is chosen based on the loss and accuracy score. We can then refit the best model with retrain option set to a boolean. If retrain is set to true it retrains the best model with the entire dataset which may lead to slight performance improvement since during fit/train process some of the dataset might have been reserved.

model.final_fit(x_train, y_train, x_test, y_test, retrain=True)

Final model can then be evaluated to measure the accuracy. It cannot be exported to a Keras model in version 0.4.0 but can be exported as an AutoKeras model that can be used within AutoKeras. The acccuracy of the model depends on the dataset as well as the time_limit to train different models.

print(model.evaluate(x_test, y_test))


We can visualise the generated models and save it as a pdf document, for which graphviz should be installed

brew install graphviz
pip install graphviz

Once thats done we can use it to generate the pdf document of the generated model

import os
from graphviz import Digraph

from autokeras.utils import pickle_from_file

def to_pdf(graph, path):
dot = Digraph()

for index, node in enumerate(graph.node_list):
dot.node(str(index), str(node.shape))

for u in range(graph.n_nodes):
for v, layer_id in graph.adj_list[u]:
dot.edge(str(u), str(v), str(graph.layer_list[layer_id]))


def visualize(path):
module = pickle_from_file(os.path.join(path, 'module'))
module.searcher.path = path
for item in module.searcher.history:
model_id = item['model_id']
graph = module.searcher.load_model_by_id(model_id)
to_pdf(graph, os.path.join(path, str(model_id)))

# path where the automated models are built from autokeras
Image classifier from autokeras

AutoKeras works well for resized images and also it has number of pre trained models for object detection, face detection and to generate and recognise voice. AutoKeras 1.0 to be released soon.

For further reading on recent research in AutoML