A Pocket History of Deep Learning

Source: Deep Learning on Medium

A Pocket History of Deep Learning

Neurons and Neural Network

Deep learning has led to unprecedented breakthroughs and innovation in many areas such as computer vision, speech recognition, fraud detection, autonomous driving and many more.

Deep learning is a specific set of techniques from the broader field of machine learning that uses artificial neural networks to learn structured representations of data, including text, images, audio and video. It is used for classifying patterns using large training data sets and multi-layer neural networks.

The origin of deep learning traces back to the dawn of artificial intelligence in the 1950s, when there were two competing visions for how to create an AI system: one vision was focused on symbolic approaches based on logic and programming, which dominated AI for decades in the last century; the other based on representation learning, where computers learn directly from data or models that automatically discover representations (for many tasks), which took much longer to develop and show tangible results.

Over seventy years of highs and lows in the development of deep learning

In 1956, John McCarthy, a Mathematics Professor at Dartmouth College proposed a workshop called Dartmouth Summer Research Project on Artificial Intelligence, which gave birth to the field of AI and motivated a generation of scientists and experts to explore the untapped potentials for computers to match the capabilities of humans \cite{sejnowski2018deep}.

In 1962, Psychologist Frank Rosenblatt at Cornell University aimed to create brain analog useful for analytical tasks. He invented a simple technique for simulating neurons in hardware and software. This marked the inception of the research in the field of enabling machines to learn and classify. Rosenblatt proposed ‘perceptron’ a single layer neural network for binary classification that could learn how to sort simple images into categories, such as, squares and triangles. Perceptron went on to be the basis of further research that culminated in the creation of multi-layer learning networks, which have formed the basis of modern deep learning. Perceptron highly resembles the modern neural network.

In the last decade, deep learning has been successfully applied to a variety of domains and applications that require large volumes of digital data (for training and extracting valuable information). This includes web searches, online maps, recommendations on e-commerce sites and streaming services, stock predictions, healthcare, and the criminal justice system, just to name a few. Recently, they have has been advancing the state-of-the-art in artificial intelligence with more powerful algorithms and optimization of hardware and software, and have led to innovative solutions in areas like robotics, healthcare, medical diagnosis — where it’s used to speed up MRI scans, deepfake detection to detect manipulated media, advances in conversational AI and assistive technology, etc.