Source: Deep Learning on Medium
What is Machine Learning?
And What is Deep Learning?
In the previous post, we discussed the definition of Artificial Intelligence (AI) that The Singularity uses. To refresh, AI is any intelligence displayed by a machine, where “intelligence measures an agent’s ability to achieve goals in a wide range of environments” (Legg and Hutter, 2007). In the same post, a definition of Machine Learning (ML) was given:
Machine Learning happens when a machine, over a period of time, gets better at a task it wasn’t explicitly told how to do.
As the reader can imagine, getting better at a task can help one to better achieve one’s goal. For example, an AI engineer could build a AI with the goal of playing Connect Four at a high level. Learning to play Connect Four could obviously help with that goal and thus with increasing the AI’s intelligence. Therefore, ML is a category within AI.
How Does Machine Learning Work?
There are different approaches to doing ML, but a very popular approach is to use an Artificial Neural Network (ANN). ANNs come in different forms. To use one for our earlier example of learning to play Connect Four, imagine the following. To play Connect Four well, one has to know what board states are better than other board states. Obviously, a winning position is good, but the position before that is (probably) also quite good because it led to a winning position. To say it differently, having four in a row is extremely good (the best), but having three in a row is also very good (in most cases). Of course, what is and what isn’t a good board position is much more complicated than this.
The ANN’s output depends on its input, but also on its internal parameters.
What you want for a Connect Four playing AI is a program that can tell how good a certain board state is: that way, the Connect Four AI can choose what move to play. That’s where ANNs come into play. An ANN can receive input (in this case a board state) and give output (in this case, a score of how good or desirable that board state is). The ANN’s output depends on its input, but also on its internal parameters. Crucially, these internal parameters can change, leading to different outputs given the same inputs.
How does an ANN change its parameters?
How do these internal parameters change? Well, the ANN can change them based on examples. If the Connect Four AI plays a lot of games of Connect Four against itself, it will be very bad at first, as the internal parameters of the ANN are not yet tuned. However, playing against itself gives the Connect Four AI a lot of examples: games will end in wins often, which the AI knows is a good thing. The ANN can adjust its parameters in order to give a relatively high score when presented with a winning board state.
Given enough games to learn from, the ANN can become a very good judge of board states.
We can tell the AI that a state before a winning state is also good (but not as good as a winning state), and again, the ANN can adjust its parameters to that. The state before the state that lead to the winning state is a little less good (probably), etc. Given enough games to learn from, the ANN can become a very good judge of board states. This is because by seeing a lot of winning board states and adjusting its parameters to give those board states a relatively high score, it will (hopefully) extract common patterns from these different board states. Common patterns could be rows of four, rows of three or rows of two of the same color.
What About Deep Learning?
Deep Learning refers to ANNs of a particular “size”. The thing is, an ANN has layers of units passing information along from the board state to the final unit determining the score of that board state. Deep Learning simply refers to ANNs that have more than three of these layers.
AlphaZero: A Deep Learning Example
AlphaZero is, in my opinion, the most amazing example of AI, and more specifically of Deep Learning, to date. An AI built by DeepMind, AlphaZero uses Deep Learning to not only be able to learn to play Chess at a superhuman level, but also two other board games: Shogi and Go. AlphaZero reached these high levels within 24 hours of playing against itself.