Perceptron: A gateway to the World of Neural Network

Original article was published on Deep Learning on Medium

Perceptron: A gateway to the World of Neural Network

Deep Learning a field so interesting yet considered to be confusing at the same time. There are so many ways to enter the world of deep learning some ways some so complicated that a person may get stuck into a scenario and decide to never explore this beautiful world.

I would like to suggest you enter this beautiful world through a gate called the Perceptron; small baby steps to get a better grip on the world of Deep Learning.

Biological Neuron and the Perceptron

Source: https://medicalxpress.com/news/2018-07-neuron-axons-spindly-theyre-optimizing.html

Us Homosapiens have always looked for inspiration, the best inspiration of all, nature. Birds inspired us to soar high through the sky, the fishes inspired us to explore the underwater world. In the year 1957, Frank Rosenblatt decided to look for inspiration within, he looked at the brain’s architecture and thought of representing the neurons as a mathematical formula leading to the birth of Perceptron.

We accomplish so many difficult tasks in the day which may seem very easy for us to think like walking, but require many calculations and passing of signals which are accomplished with the help of neurons, well they are just a part of the complex process.

Biological Neurons are connected from one end to another producing short electrical impulses which travel along the body of the cell and produces an output for the next Neuron to take action upon. Individual neurons seem to behave in a rather simple way, but they when connected to other billions of neurons in the body accomplish any difficult task quickly.

This cell and the accomplishment it attains through connection with other cells inspired us to create an Artificial Neural Network (ANN).

Perceptron

The Perceptron is the simplest ANN architecture. It is based on an artificial neuron (see fig) called a Threshold Logic Unit (TLU), where the inputs and outputs are numbers and each connection is associated with a weight. With the help of TLU a weighted sum of its inputs, i.e.,

After calculating the weighted sum(z) a step function is applied to it. The step function is usually applied to predict the output class. Step function such as Heaviside, Sign, etc is used depending upon the output expected by the algorithm.

A single perceptron can be thought of as a single biological neuron which can perform simple linear binary classification but not as much as efficient for non-linearly separable data performing complex tasks. Hence just like the biological neurons, the single perceptrons are connected in multiple layers called Multilayer Perceptron (MLP).

Multilayer Perceptron

An MLP with a single hidden layer can be represented as follows:

In simple terms, the input layer contains the dataset which is sent to all the perceptron in the hidden layer where each perceptron produces the output which is sent to the output layer as the final layer producing the output. Mathematically,

Where,

b(1), b(2) = bias vectors

W(1) , W(2) = weight matrices

G,s = activation functions

MLP’s are widely used for classification prediction problems where inputs are assigned a class or label. They are also suitable for regression prediction problems where a real-valued quantity is predicted given a set of inputs.

Conclusion

Our Deep Learning journey begins from the Perceptron going step by step you can enter the topics such as Deep Neural Network (DNN), Convolutional Neural Network (CNN), etc. The perceptron gives a brief introduction to the world of Neural Network as to how the basic component works and how such a basic component can lead a revolution in the entire industry of Neural Networks.