Artificial Neural Networks for Absolute Beginners

Original article was published on Artificial Intelligence on Medium

What is Perceptron?

An American psychologist, Frank Rosenblatt introduced a form of neural network called Perceptron as early as 1958, a machine designed for the purpose of image recognition. He predicted that the Perceptron could eventually be used to even translate languages.

A Perceptron is a single layer neural network used for binary classification problems. Similar to the biological neuron above, it also takes inputs, processes them and gives output. Being a binary classifier means that the output of the neuron can be one of the two output categories, say 0 & 1. In other words, its goal is to take two (or more) inputs and predict the output as one of the two categories/classes. Here 0 or 1 represents the two (binary) output categories we are trying to predict.

For eg:-

(a) Whether it would be a rainy day [0] or a sunny day [1] tomorrow.

(b) Whether the input image is of an animal [0] or a bird [1].

Perceptron [Image Credit: Arindom Bhattacharjee]

In the figure above, there are two inputs (x1, x2) and bias (b) going into the neuron. It performs an operation on the inputs and produces the output (y).

y = w1*x1 + w2*x2 + bwhere, 
y = output
x1 = input 1
w1 = weight associated with x1
x2 = input 2
w2 = weight associated with x2
b = bias

The machine starts off by assigning random values to these weights and biases.

w1 = 0.5
w2 = 0.5
b = -0.5

It then calculates the weighted sum. For two different set of values of x1 & x2, the weighted sum(y) is 0.5 & 1.5 respectively.

Output values for different Inputs [Image Credit: Arindom Bhattacharjee]

As can be seen, the output value can be anything depending on the input. However as described above, we need our output to be either 0 or 1. To cater to this, we introduce the activation function (i.e a step function in this case). The purpose of this step function is to check the value of the weighted sum and if this value is greater than the threshold (zero) it will return a 1, else a 0. (image is shown below).

Image Credit : Wikimedia
f(x) = 1 {if x >=0}
0 {if x < 0}

In other words, if the output is 1, the neuron is considered fired or activated, else not. Now no matter what the input values, our neuron would classify as 0 or 1 (binary classifier). In practice, there are other advanced activation functions like tanh, sigmoid, ReLU which are popularly used.