Machine Learning For Roof Detection and Solar Panel Installment

Original article was published by Harshita Chopra on Artificial Intelligence on Medium

The power of annotations

Even the most technically advanced algorithms cannot address or solve a problem without the right data. We know having access to data is quite valuable, but having access to data with a learnable structure is the biggest competitive advantage nowadays. That’s the power of data annotation.

A quirky image with hundreds of rooftops / Source:

Our wonderful team of collaborators volunteered to annotate thousands of rooftops in 500+ tiles. We pulled off a smarter method of annotating the buildings, by mapping the OSM data on the raster layer (TIF format tile) in the QGIS software.

The consistent determination of the annotators resulted in a perfectly labeled dataset for Supervised Learning algorithms.

The food for models was ready!

Scanning an image for rooftops

The major task was to detect rooftops in a given image using Computer Vision models.

Not just this, we also had to determine their type/structure such as Flat-roof, Hip-roof, Shed-roof, or any other. Hence, this became an Instance Segmentation problem.

We tried out a number of models such as Mask R-CNN, YOLACT (You Only Look At CoefficienTs), Dectectron2, and more. After training on different batches of annotations as they were delivered, we kept seeing improvement in results. Eventually, the best performing model was selected to go ahead with other tasks.


Zooming in on your rooftop

Now that we had the bounding boxes and mask contours of various rooftops, trapped properly in a dataframe, we were ready to start the analysis of individual rooftops. After extracting and zooming into masks of each detected roof, we needed the following attributes:

  • Obstacle Detection
  • Area of the roof (excluding obstacles)
  • Material of the roof
  • Detecting faces of Hip/Shed roof
  • Orientation of individual slopes

Calculating “area available” for panels

For the calculation of a rooftop’s effective area, the area occupied by obstacles has to be subtracted from the whole. So that gives rise to the task of identifying obstacles.

Due to the lack of labeled data for obstacle detection, our genius team shifted their thought process towards an unsupervised approach of edge detection and creating contours. By setting a threshold on contour colors, obstacles were distinguished from plain area to a great extent.

An effective area was therefore mathematically calculated as the difference between total area and obstacle area in terms of pixels, which was then converted into meters squared.

Roof Materials / Source:

The material of the roof

Because solar panels are installed on your home’s rooftop, it is important to understand how different roof materials may influence this process.

Generally, they range from concrete, metal, roof tiles, eternit to composite shingles.

This task also required a labeled dataset, so I decided to jump in to find a solution where we could skip annotations per se. Using Open Street Maps, we created a small but fruitful dataset labeled with roofs and their materials. A deep learning based Image Classification model was then created which identifies the material of the roof and gives the probability scores for each class.

Which way do solar panels face?


Orientation, or the direction your roof faces, may have a large impact on how productive roof-mounted solar panels will be. Your system will generate the most energy when it gets as many hours of light exposure per day as possible. In most places, the ideal power generation angle is 30–40 degrees.


The task of identifying many faces of a hip roof was a challenging one. After multiple attempts with different approaches, the task team managed to create an appreciable mathematical model that could identify the facets as well as the angles they’re inclined on, using some constructive utility functions. The output was the orientation of different roof facets.

Putting it all together

The outputs of all the tasks were captured systematically in a dataframe. Keeping in mind that we computed various attributes based on pixel values, we converted them back to geographic coordinates at the end. This allowed us to project the data on satellite images of a particular CRS (Coordinate Reference System).

After merging everything into an automated pipeline and many rounds of reviews, evaluation, fixing bugs, and testing — our software was ready to be delivered.

Solar AI is extremely happy with the final deliverables, and this is something that makes the experience even more worthwhile. As CEO Bolong Chew puts its:

“This work went beyond our wildest expectations and we’re extremely happy. We set the bar really high and the team delivered. It was an amazing experience.”


Behind the accomplishment of this goal is a dedicated, talented, and diverse team of 50 AI changemakers who devoted their precious time and effort over a span of eight weeks. We engaged ourselves in multiple brainstorming calls and weekly presentations, switching from one approach to another better one, and overcoming countless challenges. A special thanks to the task team which did a great job on the resolution improvement. This would be highly leveraged in future Omdena projects.

A big shout-out to each and every collaborator who contributed to this project!

It was once again a wonderful journey of learning and working with some of the brightest minds. Seeing our skills create an impact on the world is undoubtedly the best feeling.