Original article was published by Abdishakur on Artificial Intelligence on Medium
Deep learning for Geospatial data applications — Semantic Segmentation
A beginner’s guide and tutorial for Segmenting satellite images with Fastai
Semantic Segmentation is the process of labelling pixels or regions of the image. In Geospatial, labelling pixels for satellite images is essential in many applications including infrastructure planning, land cover, humanitarian crisis maps and environmental assessments.
Therefore, automatic segmentation, using deep learning and computer vision, can significantly help many tasks and add economic value in geospatial and earth observation domains.
In the first blog, we have covered multi-label classifications using deep learning for satellite imagery.
In this blog post, I will show the easiest way to use Deep Learning for Geospatial Applications. I will go through training a state-of-the-art deep learning model with Satellite image data. We use Fastai Version 2 built on top of Pytorch, to train our model. It will take less than ten lines of python code to accomplish this task. If you have read the first post of this series, we only need to change a few methods. That is the beautify of using Fastai.
You also do not need to worry about the Graphics Processing Unit (GPU) as we use the freely available GPU environment from Google — Google Colab. I also share the Google Colab Notebook, in case you want to interact and play with the code.
For this tutorial, we use Landcover.ai Data. The Landcover.ai semantic segmentation dataset contains 216.27 sq. km images with both 50 and 25-meter high-resolution RGB images. The images are manually annotated, which makes the data more robust.
In this dataset, we have only 3 class — Buildings, woodland and water. The fourth class is all other objects in the image, including among other field, roads, etc.
The original dataset is available at Landcover.ai website and comes with the original images and python file to process the data. In this tutorial, I will use the processed data, and it is available in this Google Drive link.
So let us get the data. First, we unzip the data from the Google Drive link and create images and labels folder. We
!unzip “/content/drive/My Drive/LandCoverData.zip”!mkdir LandCoverData/images!mkdir LandCoverData/labels
Now we need to move all JPG files to the images folder since. We also need to move all the png images to the labels since they are the masks.
!mv LandCoverData/*.jpg LandCoverData/images!mv LandCoverData/*.png LandCoverData/labels
Set up Data
To train our deep learning model, we need to set up the data. We can do this the help of Fastai Library. First, we set up the path to the image folders. Once we set up this, Fastai has a function that makes getting file names for each image easy. We call get_image_files() and provide the path we just created.
path = Path(“LandCoverData”)fnames = get_image_files(path/”images”)
We have the images, but we need to find a way to get the same labels for the image. Since we have the same names, it is only getting the same image name and adding the letter “m” and the suffix of the image which is .png.
We also need to create label names or codes. Since we have only four labels in the dataset, we can use NumPy arrays.
codes = np.array([“building”, “woodland”, “water”, “Background”])
There are different ways to create a data loader in Fastai, and since we want to create segmentation data, we can use SegmentationDataLoaders.from_label_func.
dls = SegmentationDataLoaders.from_label_func(path, bs=3, fnames = fnames, label_func = label_func, codes = codes)
We are passing the here the path, batch size which is the number of images in training per batch, fnames which is the names of the images, label function which is the labelling function we created above and final y the codes which are the labels.
Train a Deep Learning Model (Transfer Learning)
We also need to create a learner and the model. There is a lot of computations, parameters and architectures behind the scene running, but you do not need to have all the mathematical knowledge to train Convolutional Neural Network. With just two lines of code, you can run the model on your data and train it.
With multi-label classification, we used cnn_learner, but since we are dealing with segmentation, we need to use a particular type of CNN, called UNET.
learn = unet_learner(dls, resnet18)learn.fine_tune(3)
We pass here the data loader we have created from the above code and resnet18. Resnet18 is a small convolution neural network architecture that performs well in most cases. It has 11, 714, 624 trainable parameters, but that does not matter. We are using transfer learning here. This architecture is trained on another dataset, unrelated to our dataset at hand now. However, transfer learning performs well once applied to another dataset and fine-tuned to the current purpose at hand.
Once we fine-tune in the second line of code from above, the training of the data begins, and it might take several minutes depending on the environment and the dataset. The following image shows training results.
As you can see, the training loss and validation loss decreases after each epoch ( 2 epochs in total), until we finally get validation loss of 0.14. The result is not that great but good enough for beginning your journey into Deep learning segmentation. I do not want to overcomplicate things, but we need to set appropriate matrices for measuring the effectiveness of the model.
That is it! We have just trained Deep learning for semantic segmentation model using satellite imagery, with less code. We can improve the results by running more epochs, fine-tuning the model, increasing the parameters of the model, freezing layers, etc.. But I think this is just amazing and offers a great opportunity for Geo folks to run deep learning models easily.
Let us see the results. We can show results to visualize the segmented images in training.
Deep learning models are not that much complicated any more to use in any Geospatial data applications. There are a ton of resources and libraries that help you get started quickly. In this article, we have trained a deep learning model to segment four classes of land cover from satellite images with just under ten lines of code(excluding data downloading part).
I hope this article inspires you to get started using Deep learning. I am sure you have many use cases of Geospatial data applications with Deep learning.
The accompanying notebook for this article can be accessed from this link.