Understanding Clouds from Satellite Images

Source: Deep Learning on Medium

Most of the time we have 2–3 types of cloud formation in one image. 4 types of cloud formation in one image is very rare. Only one type of cloud formation in the image is also somewhat common. Furthermore, the data looks very evenly distributed for all four types of cloud formation.

Number of Cloud Types Per Image (Left Plot),Frequency of Different Clouds(Right Plot)

Now we know a little bit about the distribution of our data, we need to take a look at it and get an understanding of what it’s all about.Lets just plot a single image and a mask to get an idea of what it looks like.

Th mask is not outlining the exact clouds but roughly the area with the same kind of patterns. And from our last section, we know an image can have more than one type of cloud patterns. Lets visualize multiple clouds formations two columns. First, shows the different types of cloud formation with a bounding box. On the second column, we visualize the cloud picture with the mask segments as an overlay.

Sugar cloud formation frequently appears together in the images with Gravel or Fish cloud formation. Sugar also appears with Flower cloud formation but is less frequent. Gravels and Fish cloud formation also appears with other cloud formation. Sugar, Gravel, and Fish also appears all together in some instances.

Flower tends to occur less frequently with other clouds, and the combination of Gravel and Flower occurs but at much less frequency compared to others. In fact, Sugar, Gravel, and Fish appear all together more frequently than Grave and Flower. However, it’s not like Flower cloud formation never occurs with other cloud formation, just occurs less frequently compared to others.

In summary, they are all combination of cloud formations appearing together is a possibility, and the combinations between Sugar, Fish, and Gravel are more likely than with Flower cloud formation.

Frequent Patterns of The Cloud Formation

We first attempt to use a MaskRCNN to solve the cloud organization classification problem. It also helps to evaluate how promising the results are within the allowed processing time on Google Colab kernels as it is likely such model will take much longer to train properly. It is using the Matterplot implementation of MaskRCNN.

Architecture of the Mask-RCNN

The Data Augmentation is quite important in our case as it turns out a few samples had contrast issues and differentiating between the 4 types of clouds is no easy feat. We proceed to flip the image both vertically and horizontally, before applying different techniques to crop and alter the colours.

We initialize the model with the COCO weights even though they are quite different from the satellite imagery in the dataset provided.

We first train the heads before training the entire model .Let’s visualize training history and choose the best epoch.

The final step is to use our model to predict test data.