Original article was published on Artificial Intelligence on Medium
Brain and Artificial Neural Networks: Differences and Similarities
There is no point in reiterating that AI is on the rise. And one major contribution to this rise is the inception of artificial neural networks (ANN) which have been extensively used to make models for a variety of scenarios. Since its inception, a number of variations have been made of an ANN, each with its own utility. But with such easy implementation (what with Keras in our arsenal), we often tend to forget the roots of it. The idea of an ANN has its roots in the way the brain works, or at least that’s what their inventors claimed. ANN was developed keeping in mind the way the human brain works. There has been no dearth of research in the field of studying the brain either. We know more about our brain now more than we knew a century ago. We have come a long way since then and maybe now is the right time to check how accurate that analogy between an ANN and an actual brain is.
The correct way of doing it is to first study human behaviour. The human brain has a biological neural network which has billions of interconnections. As the brain learns, these connections are either formed, changed or removed, similar to how an artificial neural network adjusts its weights to account for a new training example. This is the reason why it is said that practice makes one perfect since a greater number of learning instances allows the biological neural network to become better at whatever it is doing. Depending upon the stimulus, only a certain subset of neurons are activated in the nervous system. The figure shows the diagram of a neuron. There is a dendritic network that takes input from other neurons and feeds it to the nucleus. The signals received are aggregated in the nucleus and if it exceeds a threshold, the axon passes a signal down to the other neurons. This is similar to the perceptron, the very first artificial neuron which just output 0 or 1 depending on whether the Heaviside function applied on the input gave a positive value or not. The neuron is either ‘activated’ or ‘inhibited’ based on the level of aggregation.
Let’s now compare it with an artificial neural network. The most obvious similarity between a neural network and the brain is the presence of neurons as the most basic unit of the nervous system. But the manner in which neurons take input in both cases is different. In our understanding of the biological neural network, we know that input is taken in from dendrites and output through the axon. These have significantly different ways of processing input. Research shows that dendrites themselves apply a non-linear function on the input before it is passed to the nucleus. On the other hand, in an artificial neural network, the input is directly passed to a neuron and output is also directly taken from the neuron, both in the same manner.
While a neuron in an artificial neural network has the capability to give a continuous set of outputs, a neuron in an artificial neural network can give only a binary output which is of the order of a few tens of millivolts. The manner in which the signal is passed and aggregated is like this. The resting potential of the neuron’s membrane is around -70mV. If the signals aggregated by the nucleus hit a certain threshold, the axon transmits a certain high voltage known as the action potential (hence the binary). For instance, if the threshold is -55mV, and each neuron gives out 5mV, a minimum of 3 neurons will have to activate this neuron for it to pass information forward. This is demonstrated by the graph below. After the threshold has been hit and the signal is passed by the axon, the cell may not be stimulated for a brief period of time known as the absolute refractory period. So there is a significant difference in the type of output that can be expected from a biological neuron and an artificial neuron.
Since transmission of information is binary in a biological neural network, research has shown that the nervous system encodes information in the frequency at which the impulses are transmitted in which the average frequency at which impulses are transmitted becomes important. Some research suggests that information is passed using temporal encoding in which the time at which the impulses are received is more important. In either case, the mechanism is starkly different from some of the techniques used to transmit information between neuron layers in artificial neural networks such as one-hot encoding of categorical information.
The learning at a very abstract level in both the systems is similar. But the method of learning is different. Artificial neural networks use gradient descent to minimize a loss function and to reach the global minimum. Biological neural networks have a different kind of learning method. Gradient descent in artificial neural networks required backpropagation which is possible only to the extent of one neuron in a biological neural network. Biological neural networks use the process of Hebbian learning using which the efficiency of one neuron being able to activate another neuron is made better by as many learning instances as possible. Spike-Dependent-Timing Plasticity strengthens the connection between neurons if input spikes in the first neuron occur immediately before output spikes in the second neuron. If the input spike of the first neuron occurs immediately after the output spike of the second neuron, the connection is weakened.
Generalization, or the ability to abstract knowledge from what one has previously learned, is an extremely useful capability that allows problem-solving across different domains quickly via minor weight adjustment — a process called fine tuning — which is a neural network’s solution to transfer learning and domain adaptation problems. The fact that not so many neuronal connections require re-wiring is one reason why if a person has one skill then it doesn’t take much time to take up a similar skill.
One major point of difference between an artificial neural network and the brain is that for the same input the neural network will give the same output but the brain may falter. It may not always give the same response to the same input and this is commonly known in the business language as human error.
Many variations have been introduced in artificial neural networks now. A convolutional neural network is one which is used to process images and each layer applies a convolution process followed by other operations on images which reduces or expands the dimensions of the image, leading the network to capture only the details that matter. The main features and computations done by convolutional neural networks were directly inspired by some of the early findings about the visual system. It was discovered that neurons in the primary visual cortex respond to specific features in the environment, like edges. Two kinds of cells were discovered: simple cells and complex cells. Simple cells responded only in a particular orientation and complex cells responded in more orientations. It was concluded that complex cells pooled over inputs from simple cells due to which there was spatial invariance in complex cells. This inspired the idea of a convolutional neural network.
Encoding models are used for predicting brain activity in response to sensory stimuli to understand how sensory information is represented in the brain. Encoding models usually apply a nonlinear transformation of stimuli to features and a linear convolution of features to responses.
There are many other differences. For instance, the number of neurons in our brain is about 86 billion. The number of neurons in normal artificial neural networks is less than 1000 which is nowhere close. Research also suggests that the power consumption by biological neural networks is around 20W whereas by artificial neural networks is around 300W.
As the gap between the human brain and neural networks closes in, the world inches towards an era of greater artificial intelligence.