Graph Learning is increasingly becoming more and more relevant as a significant amount of real-world data can be modelled as graphs.

The Graph Attention Network or GAT is a non-spectral learning method which utilizes the spatial information of the node directly for learning. This is in contrast to the spectral approach of the Graph Convolutional Network which mirrors the same basics as the Convolutional Neural Net.

In this article, I will explain how the GAT is constructed.

The basic building block of the GAT is the Graph Attention Layer. To explain the following graph is used as an example.

Here hi is a feature vector of length F.

Step 1: Linear Transformation

The first step performed by the Graph Attention Layer is to apply a linear transformation — Weighted matrixW to the feature vectors of the nodes.

Step 2: Computation of Attention Coefficients

Attention Coefficients determine the relative importance of neighbouring features to each other. They are calculated using the formula.