Source: Deep Learning on Medium
Road Crack Detection — General Workflow
How to apply deep learning algorithm to detect road crack in Indonesia.
At the plenary session of the first Esri Indonesia User Conference, EIUC 2019, Geo AI director from Esri Inc demonstrated the use of deep learning capabilities in ArcGIS to perform automated road crack detection in American road. The faster the detection process is better because road crack could cause economic and safety impact if it is not fixed immediately. In this article, I will show how to automate the detection process with use case in Indonesia.
List of contents:
1. Setting up environment
2. Prepare the data
3. Label the data
4. Train the data
Setting up evironment
Before we implement the deep learning algorithm for detecting road crack, first we need to set up the environment first. Basically we just need to install tensorflow library for training the model, but for faster training model we must to make sure that the tensorflow type is GPU instead of CPU.
Prepare the data
After we successfully set up the environment, the next thing to do is to prepare the raw data. Because we want to create a model for detecting road crack, we need a bunch of road images with road crack features as the raw data. There are two ways to get the images. First, you could take the images by taking the photos. Second, you could use the existing images which is shared publicly.
In this article, we will use the second way. I found a Youtube channel which shares a road survey video in Sumatera Selatan. Technically, we need to download the video and then split the video into several images by time (for example: if you have 60 minutes video, you could have 60 images if you divide it with a 1 minute clip rate) or frame (for example: if you have 60 minutes video which consists of 3600 frames, you could have 60 images if you use a 60 frame clip rate). *
*) Make sure that the images have road crack features.
Label the images
Next, we need to label the images by creating a box-shape boundary to the specific features, like the road crack, and then give label name to the boundary. For labelling application you could use labelImg and download that in here.
The output of the labelling process is an image (.jpg) with its label (.xml). See picture below to get a deeper understanding by what I mean.
As the deep learning model has the same concept as machine learning, to start the learning process we need to split the images dataset into train set and test set. Create two different folder named test and train, then split the results of the labelling process from the step before. We could use the 80:20 golden rule, for example if we have 100 images, put 80 images to train folder and 20 images to test folder.
After we split the images, then the next step is to train the model based on the train images. We will use Faster R-CNN configuration for the tensorflow framework as it is proven for building a good model. Set the number of steps for train the data, the bigger the steps then the results will be better. I set 50,000 steps to train the road crack model.
In order to getting understand how good is our model, we need to check the losses of the training process, the more the losses convergence to 0 then the model will be good.
For the last step, use the model we had from the previous step to the video we want to test. You could see the result from the video below.
For the next phase, I will join the result of the model in this article with GPS data and combine it Esri’s dashboard solution. This dashboard will give us information about where the crack happened and how does it look.