What AI, machine learning, deep learning actually is and what is the relationship between them?

Source: Deep Learning on Medium

Following article will give you a brief view of what artificial intelligence, machine learning, representation learning and deep learning is. How drawbacks of one are overcome by others and what is the relationship between them.

Artificial intelligence:

Artificial intelligence in computer science is the intelligence which is demonstrated by machines, in contrast to the natural intelligence which is demonstrated by humans and other animals. Basically, artificial intelligence is the discipline that covers everything related to making machines smart and intelligent. Whether it’s a robot, a car, a refrigerator, a home or a software application, if you are making them smart and intelligent, then it’s AI.

When programmable computers were first conceived, people wondered whether such machines might become intelligent, over a hundred years before one was built (Lovelace, 1842). Today, artificial intelligence (AI) is a thriving field with many practical applications and active research topics. We look to intelligent software to automate routine labour, understand speech or images, make diagnoses in medicine and support basic scientific research.

In the early days of artificial intelligence, the field rapidly tackled and solved problems that are intellectually difficult for human beings but relatively straight-forward for computers — problems that can be described by a list of formal, mathematical rules. The true challenge to artificial intelligence proved to be solving the tasks that are easy for people to perform but hard for people to describe formally — problems that we solve intuitively, that feel automatic, like recognizing spoken words or faces in images.

Knowledge-base approach to artificial intelligence:

In the knowledge-based approach, hard-coded knowledge about the world is given to the computer in formal languages. This knowledge is stored in a database called knowledge-base. A computer can reason automatically about statements in these formal languages using some logical inference rules which are placed in inference engine. This is known as the knowledge base approach to artificial intelligence. But the problem with this system is that it is not possible to feed knowledge about everything to the machine and also sometimes we don’t know how certain tasks are solved, we can perform them easily but cannot describe how to solve them and hence we cannot hard-code it to the computer.

The difficulties faced by systems relying on hard-coded knowledge suggest that AI systems need the ability to acquire their own knowledge, by extracting patterns from raw data. This capability is known as machine learning.

Machine Learning:

Machine Learning (ML) is commonly used alongside AI but they are not the same thing. ML is a subset of AI. Machine learning refers to the systems that can learn by themselves without any human intervention and these system gets smarter and smarter with time. Arthur Samuel described machine learning as: “the field of study that gives computers the ability to learn without being explicitly programmed.” Basically, ML is the mathematical and logical engine of AI.

The introduction of machine learning enabled computers to tackle problems involving knowledge of the real world and make decisions that appear subjective. A simple machine learning algorithm like linear regression and polynomial regression can tell the price of a house in a particular area. Offcourse you first have to provide them with the necessary information like the distance of the house from the highway, medical facilities in that area, number of rooms, square feet area of the house, etc. Each house will be represented in the form of some parameters and each piece of information used for representation is called a feature. In this case the number of rooms, area of a house , etc. are all features. Linear or polynomial regression algorithm will then learn how each of this feature produces different outcomes using the data provided to the algorithm for training or learning.

In the case of machine learning, we have to provide necessary features to the algorithm, the algorithm will not extract the features from the given dataset. Many AI problems can be solved by efficiently designing the features to extract for that task and feeding them to simple machine learning algorithms.

For many problems, however, it is difficult to know which features to be extracted. For example, suppose we have to write a program to detect cars in pictures. We know that cars have wheels, so we might like to use the presence of a wheel as a feature. Unfortunately, it is difficult to describe exactly what a wheel looks like in terms of pixel values. A wheel has a simple geometric shape, but its image may be complicated by shadows falling on the wheel, some metallic part of the wheel may not be visible in some images. Basically, in some cases, manual extraction of features is not possible.

Representation learning:

One solution to this problem is to use machine learning to discover not only the mapping from representation to output but also the representation itself. This approach is known as representation learning. Learned representations often result in much better performance than can be obtained with hand-designed representations.

The main disadvantage is that using representation learning we can extract only low-level features. A major source of difficulty in many real-world artificial intelligence applications is that many of the factors of variation influence every single piece of data we are able to observe. For example, suppose we want to detect faces in an image, using representation learning we are able to extract only low-level features like horizontal strokes, vertical strokes and different types of curve lines, edges, etc. We are not able to extract high-level features like presence of eye , presence of nose or presence of lips. Deep learning solves this central problem in representation learning by introducing representations that are expressed in terms of other, simpler representations. Deep learning enables the computer to build complex concepts out of simpler concepts.

Deep learning:

Deep Learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks. Neurons are the basic building blocks of the neural network. An artificial neural network consists of neurons arranged in different layers transferring information from one layer to another layer after doing some computation.

fig.Artificial neural network with 3 layers. One input, one hidden layer and one output layer.

In the above figure, each circle represents a neuron and artificial neural network consists of a large number of such neurons. The function of these neurons is similar to biological neurons. They get input from other neurons or any external source, perform some operation on that input and pass it to the neuron in the next layer. Deep learning is the field which contains neural networks with a large number of layers (here we have only three layers) at its core work.

Now let’s come to the point, how deep learning can extract high-level features which representation learning failed to extract?

fig.Neural network extracting high-level features from the image for object identity detection

Consider the above figure which shows how deep neural network extracts high-level features like ear, eye, nose and other object parts. The first layer is the input layer where we’ll feed the pixel values of the individual pixel in each neuron. At 1st hidden layer low-level feature like edges, different kind of curve lines present in the picture are detected, feeding this information or representation to the next layer, i.e., 2nd hidden layer, where higher-level features like corners and contours present in the image are detected by combining different edges detected in the previous layer. Again feeding this information to the next layer, by combining different corners and counters different object parts like eyes, ears, etc. are detected, and then a final layer after combining different parts a person or car or an animal can be detected. Thus in case of deep learning, the algorithms learn the features by itself without any human intervention when provided with a large amount of data. Usually, deep learning consists of neural networks with a large number of layers, for explanation purpose I have taken a network with only five layers.

Most AI work now involves ML because intelligent behaviour requires considerable knowledge, and learning is the easiest way to get that knowledge. Deep learning is used to learn features which cannot be manually identified and extracted. The image below shows the relationship between AI, ML, and DL.

fig.Venn diagram showing the relationship between AL, ML, RL and DL

If you want me to write an article on any specific topic please mention it in response. If you like the article do follow me and applaud, it will motivate and inspire me to write more such content related to AI. Thank you.