Original article was published on Deep Learning on Medium
Part 1. Classical Machine Learning
The first methods came from pure statistics in the ’50s. They solved formal math tasks — searching for patterns in numbers, evaluating the proximity of data points, and calculating vectors’ directions.
Nowadays, half of the Internet is working on these algorithms. When you see a list of articles to “read next” or your bank blocks your card at random gas station in the middle of nowhere, most likely it’s the work of one of those little guys.
Big tech companies are huge fans of neural networks. Obviously. For them, 2% accuracy is an additional 2 billion in revenue. But when you are small, it doesn’t make sense. I heard stories of the teams spending a year on a new recommendation algorithm for their e-commerce website, before discovering that 99% of traffic came from search engines. Their algorithms were useless. Most users didn’t even open the main page.
Despite the popularity, classical approaches are so natural that you could easily explain them to a toddler. They are like basic arithmetic — we use it every day, without even thinking.
Part 2. Reinforcement Learning
“Throw a robot into a maze and let it find an exit”
Nowadays used for:
Enterprise resource management
Popular algorithms: Q-Learning, SARSA, DQN, A3C, Genetic algorithm
Part 3. Ensemble Methods
“Bunch of stupid trees learning to correct errors of each other”
Nowadays is used for:
Everything that fits classical algorithm approaches (but works better)
Search systems (★)
Popular algorithms: Random Forest, Gradient Boosting
Part 4. Neural Networks and Deep Learning
“We have a thousand-layer network, dozens of video cards, but still no idea where to use it. Let’s generate cat pics!”
Used today for:
Replacement of all algorithms above
Object identification of photos and videos
Speech recognition and synthesis
Image processing, style transfer
Popular architectures: Perceptron, Convolutional Network (CNN), Recurrent Networks (RNN), Autoencoders
There are many more network architectures in the wild. I recommend a good article called Neural Network Zoo, where almost all types of neural networks are collected and briefly explained.