Introduction to Deep Learning

Source: Deep Learning on Medium


Deep learning is a subset of machine learning which leverages Neural Networks as the fundamental learning algorithm. They can be trained either by supervised or unsupervised learning methods. Deep learning models have been inspired by biological neurons present in our brain.

What is a neural network?

A neural network is a network of neurons connected with each other to perform a certain task. Artificial neural networks can be thought of as a graph consisting of nodes and edges. The nodes hold an input data value whereas the edges hold the weights which are used for computation purposes.

The architecture of a Neural Network.

A neural network consists of 3 types of layers:

  1. Input layer — Input to the neural network is provided using the input layers. The number of neurons present in the input layer is governed by the number of features present in the dataset.
  2. Hidden layer — These are the layers where the computations take place within the neural network. The number of hidden layers and the neurons present in layers respectively varies from task to task and is often empirically determined.
  3. Output layer — Output of the neural network is extracted from the output layers. The number of output neurons depends on the number of output classes present in the dataset.

Applications of Deep Learning

Deep learning architectures have been employed in various domains like Computer Vision, Natural Language Processing, Machine Translation, Speech Recognition etc and have shown remarkable performance as compared to traditional machine learning methods. Let’s look at the various different types of deep learning architectures that are used for different tasks.

  1. Simple classification — Deep learning can be employed for classification tasks wherein given a data point, we have to predict the class that data point belongs to. For example, let us consider the task of predicting the stage of cancer. In this case, the input to the neural network could be features like the gender of the patient, age of the patient, size of the tumour, age of the tumour etc and the output would be the stage of cancer. Simple feed-forward neural networks are used for this task.
  2. Online Advertising — This is a very lucrative use case of deep learning wherein we have to identify advertisements that a user might view. This is used for revenue-boosting and identifying potential users of a particular product. Here, the input to the neural network could be the advertisement and user information and the output would be the choice that the user might choose i.e either view the advertisement or dismiss it. Simple feed-forward neural networks are used for this task.
  3. Photo Tagging — Given an input image, the task is to predict the class that the image belongs to. With advancements in deep learning research, this task has become finer by predicting in the image, where a particular object is present (Object Detection) and even finer by predicting which pixels belong to a particular object (Semantic segmentation). Convolutional neural networks are used for this task.
  4. Speech Recognition — It is an important use case for deep learning applications. In this case, the input to the neural network is an audio file and the output would be text transcript of the input audio file. Recurrent neural networks are used for this task.
  5. Machine Translation — Deep learning architectures perform remarkably well is the task of machine translation. The input to the neural network is a textual document in a particular language and the output is the corresponding text in a different language. For example, translating a text in Hindi to English, Arabic to Mandarin etc. Recurrent neural networks are used for this task.

Why is Deep Learning taking off now?

Deep learning algorithms were first developed in the 1970s. But back in those days, there were many hindrances to practical Deep Learning. Over time until the last decade, these hindrances faded and the power of Deep Learning was slowly realized. Let us discuss these hindrances in detail.

1. Data — Deep learning architectures are data hungry; the more data they are provided with, the better they perform. According to an article published in 2013 by SINTEF, 90% of the world’s data was generated over the past 2 years. This data was previously not present for the data-hungry algorithms and posed a threat to their performance. Today, we generate 2.5 quintillion bytes of data every day which satiates Deep Learning architectures and produce remarkable results.

2. Computation — Moore’s law is the observation that the number of transistors in dense integrated circuit doubles about every two years. This increase in the number of transistors has increased the speed of computations and reduced the processing time of algorithms. This helps people to code more, experiment more and ideate more. A lot of advances in deep learning has been brought about due to the faster iteration of code, experiment and idea cycle.

3. Algorithm — Hardware alone cannot bring out significant improvement until the software also improves. The number of annually published papers in the field of Artificial Intelligence has been growing exponentially over the past decades. These improvements are in the field of optimization of architectures, their regularization, the learning strategies used and different new types of architectures. Coupled with their applications in different domains, the improvements in deep learning algorithms are pushing their boundaries.