Source: Deep Learning on Medium
Artificial Intelligence (AI) is penetrating into every aspect of our life. From smart assistants guiding us around town to recommendation engines guessing what products we should place in our shopping cart or what song should we listen to next.
The progress has been rapid, impacting every industry on the planet — from the most traditional ones like agriculture to the emerging disciplines like robotics. The vast majority of this rapid progress can be attributed to advances in deep learning — the novel approach to creating the most “human-like” artificial intelligence.
What is Deep Learning (DL)?
Deep learning is a subdivision of machine learning with a strong emphasis on teaching computers to learn like humans: by being presented with an example.
As a child, you easily learn how an apple looks — the shape, the color, the texture — and you learn to understand that when you hear the word “apple”, you will likely receive a sweet, round red object that you can bite into.
Deep learning methodologies operate on the premises that machines can be taught from experience as well. The algorithm is presented with the same task repeatedly, and each time receives feedback on its performance so it could adjust its accuracy in the future.
How Deep Learning is Different from Machine Learning
Deep learning is a subdivision of machine learning. Both disciplines pursue the same goal — teach machines to become “smarter” in what they do.
Machine learning algorithms can be programmed to perform accurate tasks — classify data, predict prices and so on. As new data becomes available, their performance improves. However, those improvements and adjustments need to be performed by a human engineer.
Deep learning reduces the level of human involvement in the teaching process. Such algorithms are only given data and the initial parameters for operationalizing that data. They are capable to determine whether their output (prediction or action) is accurate or not on their own.
Let’s illustrate this with a quick example. You have a voice-controlled thermostat, programmed to adjust the temperature whenever you activate it and say “20℃”. If it’s powered by machine learning, over time it can learn to capture the digit component in more complex commands — e.g., “Please, make it 20℃ at home”.
Now, if your thermostat is powered by a deep learning model, over time it could figure out to start adjusting the temperature whenever it hears something like “Gosh, it’s cold!” or “I’m freezing today!”. In essence, it’s capable to learn using its own “brain” or more precisely — an artificial neural network.
What is a Neural Network?
Deep learning and neural networks are the pillars for building the new generation of intelligent solutions.
Artificial neural networks
Artificial neural networks are algorithmic representations of biological neural networks, which are powering different cognitive processes inside the human brain — vision, hearing, decision-making. Artificial networks can learn from a large volume of data, by example with little-to-no supervision.
There’s not much of a difference between deep learning and neural networks, as the latter is the baseline method of DL. Deep learning assumes using a subset of neural networks to accomplish various tasks. The term “deep” was added exactly due to the fact that artificial neural networks come with a varying number of (deep) layers, powering the learning process.
So, how do neural networks work? In short, each ANN consists of “artificial neurons” — mathematical functions that analyze incoming data and transmit it to the next “neuron” for further analysis. Every layer in the network focuses on analyzing specific features, e.g., shadows in edges in number 1 for written digit recognition tasks, before passing on the “knowledge” to another layer that will perform further operations with it before delivering the result, e.g., recognizing number 1 as one.
To further understand how neural networks function, let’s take a closer look at the common types of neural networks developed up to date.
Feed forward neural networks
Feed forward neural networks are the most “simple” type of an artificial neural network, first proposed in 1958 by AI pioneer Frank Rosenblatt. Within such network, information travels only one-way — from left to right, through the input nodes, then through the hidden nodes (if any) and afterwards through the output nodes.
Each node in the layer is an artificial neuron — represented by a function that performs required calculations for the task at hand, e.g., classify the data based on a certain parameter. To move from the Input layer (when data or features are provided to the network) to Output layer (which delivers the prediction) — different linear or nonlinear functions are applied.
Hidden layers enable the computation of more complex functions by cascading simpler functions. In other words, a network with no hidden layer (a simple artificial neuron) is only capable of learning a linear decision boundary, i.e., classify all the blue dots to one side of the decision boundary and all the red ones to the other side, but it will not be able to handle more complex decisions.
Thus, hidden layers enable additional learning capabilities that can tackle more complex decisions. Neural networks further vary depending on the type of hidden layers used.
Recurrent neural networks (RNNs)
In this case, the input information travels through a loop. Before producing a decision, the network will take into consideration the current input, plus the data it has captured from the previously operationalized inputs.
RNNs come with a short-term memory. They are aware of the recent past as it produces a certain output, “remembers it” and loops it back into the network. This memory feature makes RNNs highly effective for tasks such as speech and text recognition; financial data analysis and predictions; and more. Unlike other algorithms, they have a deeper understanding of a sequence and its context. This way they produce predictive results in sequential data that no other algorithm can muster.
Convolutional neural networks (CNNs)
Convolutional neural networks are the closest technical similitude to the brain we have managed to develop so far. These deep artificial networks attempt to closely mimic the processes running in our primary visual cortex, responsible for our ability to “see” and “recognize” objects. Thus, CNNs are mainly used for image/video recognition tasks.
CNNs differ from the other two types of networks. The layers are organized in 3 dimensions: width, height and depth. This structure enables better recognition of different objects.
As you can see above, this example of neural network has two distinctive components:
- Hidden Layers — at this stage, the network performs a series of operations trying to extract and detect specific image features. If you have a picture of a car, the network will learn how a wheel, a door, a window looks like.
- Classification — additional layers serve as a classifier on top of the extracted features. These layers will determine the probability of how likely the image is being what the algorithm predicts it is.
Infopulse team has recently worked on interesting CNN project with a client in metal manufacturing niche. The company needed a new solution for automating readings of old gauge equipment as their step towards the manufacturing Industry 4.0, powered by IoT and increased digitalization. To rapidly capture and process that data, our team has created a convolutional neural network, capable to perform image recognition in less than 2 seconds on average on those small, outdated devices. Learn more about our solution from this case study.
So, what are neural networks capable of in the business setting apart from classifying data and recognizing patterns?
- Continuous estimation: NNs are the perfect candidates for dealing with time series data prediction. For example, by being given historic sales figures, current market trends, weather and consumer sentiment, a network can estimate the demand for winter boots or another seasonal product or service.
- Clustering: create different customer segments based on demographic data, online behavior, and preference data from individual consumers.
- Process optimization: estimate the best route for a fleet based on different parameters such as time and fuel usage.
- Advanced recommendations: suggest the “song you will like” or “product to buy” with an accuracy higher than traditional predictive analytics algorithms.
- Anomaly detection: a network could be trained to recognize different trading patterns and estimate when a spike or a crash is most likely to occur.
- Data generation: after being presented with a series of artwork, a neural network can learn how to produce new pieces in similar styles. This year Christie’s has sold the first piece of AI art for $432,500.
Continue reading this post on the Infopulse blog where it’s been originally published.