Building a Plant disease classification web app in Keras and Tensorflow.js (PART 2)

Original article was published by Similoluwa Okunowo on Deep Learning on Medium


Building a Plant disease classification web app in Keras and Tensorflow.js (PART 2)

Hi friend, this is the PART 2 of the two-part tutorial series on building a web app with Keras and Tensorflow.js. I encourage you to read the first part HERE.

LINK TO FIRST PART :-

https://medium.com/@rexsimiloluwa/building-a-plant-disease-classification-web-app-in-keras-and-tensorflow-js-d435829213fa

In the first part of this tutorial, i showed you how to build/train a MobileNet model to classify diseases in images of plants (crop leaves) downloaded from Kaggle (using Kaggle API). We did this by employing the concept of transfer learning. I genuinely hope you learnt alot from that part.

Deploying machine learning models for public use is indeed essential to actually creating value with Machine learning. Therefore, in this final section, i will teach you how to deploy the already trained model using Tensorflow.js and Javascript in a web application for browser-based inference.

This will be achieved by converting the python model to a Javascript compatible format, using tensorflowjs-converter (A highly efficient python module which converts saved keras models to formats compatible with Javascript). Please refer to the Tensorflow.js Documentation here for analysis of its advanced features, use-cases and tutorials.

This is a demo of the final application we will be building (Please ignore the UI ) :-

Requirements / Pre-requisites :-

  • Google Colab (We used this extensively in the first part, I assume you have this set-up already),
  • The previously saved MobileNet model from the first part of the tutorial (mobilenet_model.h5),
  • Basic knowledge of HTML, CSS, JavaScript (This is not very neccesary however),
  • An IDE for writing HTML, CSS and Javascrip codes (i.e VSCode, Sublime Text etc.),
  • And most importantly, the willingless to learn.

So, let’s get started.

Codes and Implementation

The codes and implementation will be explained in well-detailed steps to preserve structure and aid understanding.

STEP 1 :- Converting the Keras Model to a Tensorflow.js compatible model

This is the initial and most important step. This is achieved using a Tensorflow.js converter module in Google colab which converts our saved model (in HDF5 or .h5 format) to a .json format which is compatible with a Javascript environment. The Tensorflow.js converter also works with several other file formats such as Tensorflow SavedModel format, Tensorflow Hub module e.t.c.

To use this converter, we need to install it in our Google Colab environment, Kindly run this command in a new code cell in Colab :-

# This downloads/installs tensorflowjs into your Google colab environment !pip install tensorflowjs

Next up, create a new folder in your base directory (i.e PLANT DISEASE RECOGNITION folder) where the converted Tensorflow.js models will be stored :-

Next up, we can easily convert the keras model to a tensorflow.js model using the tensorflowjs_converter command. The arguments for the tensorflowjs_converter are the input_format (keras),directory containing the saved model (‘models/mobilenet_model.h5’), and the output directory for the converted model (‘tensorflowjs-model/’) Please note that this can also be done in any IDE or environment of your choice. Run this command in a new code cell :-

!tensorflowjs_converter --input_format keras models/mobilenet_model.h5 tensorflowjs-model/

Voila !, Our model has been successfully converted, and the output files are stored in a zipped file in the specified output directory (‘tensorflowjs-model/’). We can easily unzip this and download directly to our Local machine (computer) :-

Great work so far, we have successfully downloaded our converted model. The ‘model.json’ file is the main file which will be used to load the model on a browser to make predictions. Please download the folder ‘tensorflowjs-model’ folder and ‘class_indices.json’ file from your Google Drive into your Local machine.

STEP 2: Creating the Frontend for the Web application using HTML and CSS

This section involves creating the frontend for the web application. I won’t really walk you through the steps involved in creating the frontend explicitly, as that is not the core objective of this tutorial. However, only simple HTML elements and CSS styles will be used.

Folder Structure for the Project :-

Please ensure that that tensorflowjs-model folder containing the downloaded model.json file is stored in your working directory locally, and create an HTML file ‘index.html’, a CSS file ‘style.css’, and a JS file ‘script.js’ which will contain the Javascript codes:-

root /
PLANT DISEASE CLASSIFIER APP
|-- tensorflowjs-model
|-- index.html
|-- style.css
|-- script.js
|-- class_indices.json

The major HTML element used is the <input/> element for uploading image files to the browser for prediction. The Basic structure of the HTML webpage is shown below (It’s also well-commented for beginners) :-

‘style.css’ — The external CSS Stylesheet containing the simple styling for the website (I encourage you to tweak this to your preferred taste) :-

Great work so far, you can open the index.html file from your IDE/ Text editor in your browser to see the interface.

Next, we need to Load Tensorflow.js library from cdn.js into our HTML file(A content delivery network) to run the model in the browser, and make predictions. Include this <script></script> tags just above the </body> tag to load Tensorflow.js, and the external Javascript file containing the JS codes.

<script src="https://cdnjs.cloudflare.com/ajax/libs/tensorflow/2.0.1/tf.min.js"></script><script src="script.js"></script>

STEP 3 :- Writing the functionality in Javascript

This section entials implementing the entire Javascript functionality shown in the Demo at the beginning of the article. Summarily, we want to to read the uploaded image from our browser, Convert the uploaded image to a Tensor/Array, Load our Tensorflow.js model, perform required scaling operations and transformations on the uploaded image, and generate predictions.

The first step is to initialize/load the model which was saved in the ‘tensorflowjs-model’ folder. We will be using Javascript promises (async/await) to deal with asynchronous operations in a sequential/fancier manner/syntax :-

Next up, we apply the initialized model on the transformed image in the browser to generate predictions.

The ‘fetchData()’ async function simply fetches the json file containing the class_indices, and parses it to a Javascript Object. The class_indices.json file contains the encoded class index and label names. This is the object shown in the Javascript console :-

The predict() function simply loads the uploaded image, applies the required transformations on the image (i.e re-scaling), and generates the predictions.

Then we need to compute the index of the Maximum value in this array (using tf.argMax() method). The output is the class with the highest probability score because we used the ‘softmax’ activation function while compiling our model. This index can then be used to get the class name from the class_indices Object, which is then displayed in the browser. The percentage of accuracy is also displayed. Please, i encourage you to read through the following steps and relate them to the codes written above.

STEP 4 :- Hooking it all up

Finally, it is time to hook it all up and make predictions on our browser. I am really proud of you, and grateful if you read thus far. We use Event Listeners in Javascript to invoke the initialize() and predict() async functions when an image is uploaded from the <input> element, This is the complete Javascript code.

Voila !, you have successfully converted your Python Keras model to a Tensorflow.js model to make inferences in yout browser. I’d implore you to take your time to understand the entire process, and then improve the functionality.

This project can be deployed for public use on Github pages for free, Netlify or any other hosting service. My version of the project can be accessed via this URL :- https://bit.ly/3k2MASg

Learning Objectives Achieved

Congratulations friend, for learning something new and useful. I really the two tutorials were informative enough and well-structured to aid understanding. We have completed the following learning objectives in the PART 1 and PART 2 of the tutorial series :-

  1. Understanding how to download public datasets from Kaggle using Kaggle API,
  2. Understanding how to build a transfer learning model with MobileNet and Keras, and train the model on the downloaded dataset seamlessly,
  3. Understanding how to Convert Python Keras models to Javascript compatible versions (using Tensorflow.js converter),
  4. Understanding how to deploy the model as a web application for inference on any browser.

There are a lot of ways to harness the knowledge gained from these tutorials to build other useful solutions. This is only a basic use-case, and i really encourage you to improve the functionality and optimize the solution.

Link to Github Repository :-

Please connect with me on LinkedIn if you have any questions or contributions.

Kindly leave claps to acknowledge my effort. I’d genuinely appreciate that, Thanks for reading Friend !

Stay safe folks, and keep harnessing useful knowledge. Thanks for reading !