Road Crack Detection — General Workflow

Source: Deep Learning on Medium

Road Crack Detection — General Workflow

How to apply deep learning algorithm to detect road crack in Indonesia.

At the plenary session of the first Esri Indonesia User Conference, EIUC 2019, Geo AI director from Esri Inc demonstrated the use of deep learning capabilities in ArcGIS to perform automated road crack detection in American road. The faster the detection process is better because road crack could cause economic and safety impact if it is not fixed immediately. In this article, I will show how to automate the detection process with use case in Indonesia.

List of contents:
1. Setting up environment
2. Prepare the data
3. Label the data
4. Train the data
5. Simulation

Setting up evironment
Before we implement the deep learning algorithm for detecting road crack, first we need to set up the environment first. Basically we just need to install tensorflow library for training the model, but for faster training model we must to make sure that the tensorflow type is GPU instead of CPU.

I will not explain in detail how to install the environment, but if you want to learn more about it, you could check this link:
Link 1
Link 2

Prepare the data
After we successfully set up the environment, the next thing to do is to prepare the raw data. Because we want to create a model for detecting road crack, we need a bunch of road images with road crack features as the raw data. There are two ways to get the images. First, you could take the images by taking the photos. Second, you could use the existing images which is shared publicly.

Example of road image with road crack (marked with red circle)

In this article, we will use the second way. I found a Youtube channel which shares a road survey video in Sumatera Selatan. Technically, we need to download the video and then split the video into several images by time (for example: if you have 60 minutes video, you could have 60 images if you divide it with a 1 minute clip rate) or frame (for example: if you have 60 minutes video which consists of 3600 frames, you could have 60 images if you use a 60 frame clip rate). *

*) Make sure that the images have road crack features.

Label the images
Next, we need to label the images by creating a box-shape boundary to the specific features, like the road crack, and then give label name to the boundary. For labelling application you could use labelImg and download that in here.

Labelling process with labelImg application

The output of the labelling process is an image (.jpg) with its label (.xml). See picture below to get a deeper understanding by what I mean.

Image (.jpg) with its label (.xml)

Train model
As the deep learning model has the same concept as machine learning, to start the learning process we need to split the images dataset into train set and test set. Create two different folder named test and train, then split the results of the labelling process from the step before. We could use the 80:20 golden rule, for example if we have 100 images, put 80 images to train folder and 20 images to test folder.

Test set
Train set

After we split the images, then the next step is to train the model based on the train images. We will use Faster R-CNN configuration for the tensorflow framework as it is proven for building a good model. Set the number of steps for train the data, the bigger the steps then the results will be better. I set 50,000 steps to train the road crack model.

Training process with step number information

In order to getting understand how good is our model, we need to check the losses of the training process, the more the losses convergence to 0 then the model will be good.

Losses of the training process

Test model
For the last step, use the model we had from the previous step to the video we want to test. You could see the result from the video below.

For the next phase, I will join the result of the model in this article with GPS data and combine it Esri’s dashboard solution. This dashboard will give us information about where the crack happened and how does it look.

Road crack dashboard
Road crack dashboard