Source: Deep Learning on Medium
This is the first post in Deep Learning for Humans series.
We humans always felt in love with the things we created. The dates can go back to ancient Greeks, where Pygmalion — the Greek artist felt in love with Galatea, the sculpture he created and wished to give his creation a life.
When programable computers were first conceived, people wondered if they can also think and behave like we humans. Such crazy questions led to the foundation of a new era which led to the revolution in Information Sciences and the modern day Artificial Intelligence. Today, AI is a thriving field with many practical applications and active research topics. We look to intelligent software to automate routine labour, understand speech or images, make diagnoses in medicine and support basic scientific research.
In the early days of AI, the field rapidly tackled and solved problems that are intellectually difficult for human beings but relatively straightforward for computers — problems that can be described by a list of formal, mathematical rules. The true challenge to artificial intelligence proved to be solving the tasks that are easy for people to perform but hard for people to describe formally — problems that we solve intuitively, that feel automatic, like recognising spoken words or faces in images.
Here in this coming series of blog posts, we will study the solutions to these more intuitive problems. This solution is to allow computers to learn from the experiences and understand the world in terms of a hierarchy of concepts, with each concept defined through its relation to simpler concepts like we humans do. By gathering knowledge from experience, this approach eliminates the need of human operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts also helps the computer to learn complicated concepts by building them out of simpler ones. If we have to visualise this process, or draw a graph showing how these concepts are built on the top of each other, the graph comes out to be deep, with many layers. For this reason, we call this approach to AI deep learning.
Many of the early success of AI took place in relatively sterile and formal environments and did not require computers to have much knowledge about the world. For example, IBM’s Deep Blue chess playing system defeated world champion Garry Kasparov in ’97. Chess is of-course a very simple game in terms of computers which only have sixty-four locations and thirty-two pieces that can move in only rigidly circumscribed ways. Devising a successful chess strategy is a tremendous accomplishment, but the challenge is not due to the difficulty of describing the set of chess pieces and allowable moves to the computer. Chess can be completely described by a very brief list of formal rules, easily provided ahead of time by the programmer.
Computers are really good in doing computations. Ironically, abstract and formal tasks that are among the most difficult undertakings for a human being are among the easiest for a computer. Think of a calculator defeating human in basic mathematical operations — exactly abstract and formal task. A person’s everyday life requires an immense amount of knowledge about the world. Majority of this knowledge is subjective and intuitive, and therefore difficult to articulate in a formal way. Computers need to capture this same knowledge in order to behave in an intelligent way. One of the fundamental challenges in AI is to get this informal knowledge into a computer.
Several AI projects have sought to hard-code knowledge about the world in formal languages. A computer can reason automatically about these statements in these formal languages using logical inference rules. This is known as the knowledge based approach to artificial intelligence. None of these projects has led to major success. The difficulties faced by systems relying on hard-coded knowledge suggests that AI system need the ability to acquire their own knowledge, by extracting patterns from raw data. This capability is known as machine learning. Machine learning enabled computers to tackle problems involving knowledge of the real world and make decisions that appear subjective. A simple machine learning algorithm called logistic regression were able to determine whether to recommend cesarean delivery (Mor-Yosed et al., 1990). A simple machine learning algorithm called naive Bayes can separate legitimate emails from junk emails or spam emails.
The performance of the given algorithm is directly proportional to the quality of data they are provided. For example when logistic regression is used to recommend cesarian delivery, an AI system doesn’t examine the patient directly. Instead, the doctor tells the system several pieces of relevant information such as presence or the absence of uterine scar. Each piece of information provided to the AI system is called as a feature. Logistic regression learns how each of these features can correlate with the various outcomes. You can imagine teaching a child various features that constitute the letter ‘A’, and asking him to tell that a given letter is ‘A’ or not on the basis of what he has learnt. This dependence on representations is a general phenomenon that appears throughout computer science and even daily life. In computer science, operations such as searching a collection of data can proceed exponentially faster if the collection is structured and indexed intelligently. People can easily perform arithmetic on Arabic numerals but ﬁnd arithmetic on Roman numerals much more time consuming. It is not surprising that the choice of representation has an enormous eﬀect on the performance of machine learning algorithms.
Many artiﬁcial intelligence tasks can be solved by designing the right set of features to extract for that task, then providing these features to a simple machine learning algorithm. For example, a useful feature for speaker identiﬁcation from sound is an estimate of the size of the speaker’s vocal tract. This feature gives a strong clue as to whether the speaker is a man, woman, or child. For many tasks, however, it is difficult to know what features should be extracted. For example, a useful feature for speaker identification from sound is an estimate to the size of the speaker’s vocal tract. This feature gives a strong clue as to whether the speaker is a man, women or a child. But in some cases, we don’t know what kind of feature should be taken into consideration. For example, consider a program that detects cars in photographs. We know that cars have wheels, so we might like to use the presence of wheels as a feature. Unfortunately, it is difficult to describe exactly what a wheel looks like in terms of pixel values. A wheel has a simple geometric shape, but its image may be complicated by shadows falling on the wheel, the sun glaring oﬀ the metal parts of the wheel, the fender of the car or an object in the foreground obscuring part of the wheel, and so on.
One solution to this problem is to use machine learning to discover not only the mapping from representation to output but also the representation itself. This approach is called representation learning. Learned representations often result in much better performance than can be obtained with hand-designed representations. They also enable AI systems to rapidly adapt new tasks, with minimal human intervention. A representation learning algorithm can discover a good set of features for a simple task in minutes, or for a complex task in hours to months. Manually, designing features for a complex task requires a great deal of human time and effort; it can take decades for an entire community of researchers.
Deep Learning solves this central problem in representation learning by introducing representations that are expressed in terms of other, simpler representations. Deep learning enables the computer to build complex concepts out of simpler concepts.
The above figure shows how a deep learning system can represent the concept of an image of a person by combining simpler concepts, such as corners and contours which are in turn defined in terms of edges.
Wrapping this up, deep learning is an approach to machine learning that has drawn heavily on our knowledge of the human brain, statistics and applied math as it developed over the past several decades. In recent years, deep learning has seen tremendous growth in its popularity and usefulness, largely as the result of more powerful computers, larger datasets and techniques to train deeper networks. The years ahead are full of challenges and opportunities to improve deep learning even further an to bring it to new frontiers.