Deep learning for Geospatial data applications — Multi-label Classification

Original article was published by Abdishakur on Deep Learning on Medium


Deep learning for Geospatial data applications — Multi-label Classification

A beginner’s guide and tutorial for classifying satellite images with Fastai

Photo by USGS on Unsplash

Artificial intelligence (AI) and Machine learning (ML) have touched on every possible domain and the Geospatial world is no exception. The intersection of Geospatial data and AI is already disrupting many industries and holds great potential in many Geospatial driven applications including agriculture, insurance, transportation, urban planning and disaster management.

Classifying, detecting or segmenting multiple objects from satellite images is a hard and tedious task that AI can perform with more speed, consistency and perhaps more accurate than humans can perform. We call this Computer vision, and in particular, a subtype of machine learning called Deep Learning (DL) is disrupting the industry. Deep learning, an algorithm inspired by the human brain using Neural networks and big data, learns (maps) inputs to outputs.

However, Neural networks require a large number of parameters and fine-tuning to perform well and not in the distant past using neural networks required building a large number of parameters from scratch. In addition, Graphics Processing Unit (GPU) availability was limited, which is crucial for doing deep learning. With current advances in technology and the availability of GPUs, we can use transfer learning to apply Deep learning with any imaginable domain easily without worrying about building it from scratch.

In this tutorial, I will show the easiest way to use Deep Learning for Geospatial Applications. I will go through training a state-of-the-art deep learning model with Satellite image data. We use Fastai Version 2 built on top of Pytorch — to train our model. It will take less than ten lines of python code to accomplish this task. You also do not need to worry about the Graphics Processing Unit (GPU) as we use the freely available GPU environment from Google — Google Colab. I also share the Google Colab Notebook, in case you want to interact and play with the code.

The Data

For this tutorial, we use UCMerced Data, the oldest and one of the popular land-use imagery datasets. This data consists of 100 images per class with 21 land use classes. The land use classes for this dataset are:

agricultural  forest         overpass      airplane    freeway   parkinglot    runway         golfcourse    river       beach harbor                    buildings     intersection   storagetanks chaparral    tenniscourt mediumresidential   denseresidential       mobilehomepark 
baseballdiamond sparseresidential

The following image shows random images with class names from UCMerced dataset.

Sample of UCMerced Land use dataset — Image generated by the author.

To get the data we can use wget functionality to directly download the data. Here, I am using Google Colab Jupyter Notebook, but this will work with any Jupyter Environment.

!wget [<http://weegee.vision.ucmerced.edu/datasets/UCMerced_LandUse.zip>](<http://weegee.vision.ucmerced.edu/datasets/UCMerced_LandUse.zip>)
!unzip UCMerced_LandUse.zip

With just these 2 lines of code above, we access the data, download it and unzip it. And we are ready for doing some Deep learning stuff. The data consists of 21 folders with each class in the dataset under one folder name ( See the image below).

Set up Data

To train our deep learning model, we need to set up the data. We can do this the help of Fastai Library. First, we simply set up the path to the image folders. Once we set up this, Fastai has a function that makes getting file names for each image easy. We call get_image_files() and provide the path we just created.

# 1. Set up the path to the image folders
path = Path("/content/UCMerced_LandUse/Images")
# 2. Get images using get_image_files() function
items = get_image_files(path)

Now, we need to create a DataBlock and load the data to Pytorch. Again we can do this with just two lines of code.

# 1. Use DataBlock API to get the data.
data = DataBlock(blocks=(ImageBlock, CategoryBlock), get_items=get_image_files, splitter=RandomSplitter(valid_pct=0.3, seed=42), get_y=parent_label, item_tfms=Resize(128))
# 2. Load the data with dataloaders
dls = data.dataloaders(path, item_tfms=Resize(128))

The first line takes care of getting images from folders, splitting them between training and validation datasets and mapping the labels from the filenames in the folders. The second line loads the data and resizes them into an image of 128 by 128 pixels, we call this dls. In the next section, we train a state of the art deep learning model for the geospatial data, classifying satellite imagery into 21 different land use classes, again with only two lines of Python code.

Note that DataBlock API is a High-level API to quickly get your data into data loaders. The Fastai library also provides lower-level APIs to offer greater flexibility to most of the datasets types used (i.e, from CSV or Dataframe)

Train a Deep Learning Model (Transfer Learning)

To train our Deep learning model, we need to create a learner and the model (with fine-tuning it). There is a lot of computations, parameters and architectures behind the scene running, but you do not need to have all the mathematical knowledge to train Convolutional Neural Network. With just two lines of code, you can run the model on your data and train it.

# 1. Create a learner 
learn = cnn_learner(dls, resnet18, metrics=error_rate)
# 2. Run the model
learn.fine_tune(5)

The first line of code above creates a learner. Here, we provide the data loader we create earlier. We also need to choose the deep learning architecture we want to use. Resnet18 is a small convolution neural network architecture that performs well in most cases. It has 11, 714, 624 trainable parameters, but that does not matter. We are using transfer learning here. This architecture is trained on another dataset, unrelated to our dataset at hand now. However, transfer learning performs well once applied to another dataset and fine-tuned to the current purpose at hand.

Once we run the model in the second line of code from above, the training of the data begins and it might take several minutes depending on the environment and the dataset. The following image shows training results

As you can see, the training loss and validation loss decreases after each epoch ( 5 epochs in total). More importantly, the error rate is our metric and shows the rate/percentage of error in each epoch(iteration). Our last error rate indicates to be around 0.080 (or in terms of accuracy 92% accurate).

That is it! We have just trained a Deep learning model using Geospatial data and got an accuracy of 92% without writing that much code. We can improve the results by running more epochs, fine-tuning the model, increasing the parameters of the model, freezing layers etc.. But I think this is just amazing and offers a great opportunity for Geo folks to run deep learning models easily.

Interpretation of the Model

A confusion matrix is a great visual way to interpret how your model is performing. The confusion matrix compares the predicted class with the actual class. We can create a confusion matrix like this.

# 1. create classificaiton interpretation
interp = ClassificationInterpretation.from_learner(learn)
# 2. Plot a confusion matrix
interp.plot_confusion_matrix(figsize=(12,12))

The following diagram shows the confusion matrix of the dataset. The most confused classes are the three different types of residential classes: dense residential, medium residential and sparse residential. This is obviously an issue of where to put the boundary line between these three different types of classes. In general, the model performs well with 1 or 2 misclassified images per class.

Note that the confusion matrix is just one method of model interpretation. It applies only on single-label classification like our dataset.

Inference

In order to use other images and classify them, you can use your trained model to predict them. This is simply calling learn.predict() and providing the image you want to classify.

# Predict the image class
learn.predict("/content/geoeye-1-kutztown.jpg")[0]

The output is a prediction of the class. In this case, our model predicts correctly that that is an airplane. You can try other images and find out how the model generalizes to other unseen images.

Note that this is a single-label classification problem, but in most cases you have probably multi-label classification where images have different objects. Training Multi-label classification is not much different from the single-label classification we have done and only requires to use another DataBlock for multicategory applications.

Conclusion

Deep learning models are not that much complicated any more to use in any Geospatial data applications. There are a ton of resources and libraries that help you get started quickly. In this article, we have trained a deep learning model to classify land use from satellite images with just under ten lines of code (excluding data download and zipping part).

I hope this article inspires you to get started using Deep learning. I am sure you have many use cases of Geospatial data applications with Deep learning.

The accompanying notebook for this article can be accessed from this [link](https://colab.research.google.com/drive/16N8cEPPbGqZq3YWQnFxsuQS-ysARnGNc](https://colab.research.google.com/drive/16N8cEPPbGqZq3YWQnFxsuQS-ysARnGNc)):