The Games That AI Won

Original article can be found here (source): Artificial Intelligence on Medium

And The Progress They Represent

Some tasks that AI does are actually not impressive. Think about your camera recognizing and auto-focusing on faces in pictures. That technology has been around since 2001, and it doesn’t tend to excite people. Why not? Well, because you can do that too, you can focus your eyes on someone’s face very easily. In fact, it’s so easy you don’t even know how you do it. If AI can do it too, then who cares how it works? Though we may not explicitly understand how this AI works, its underlying mechanisms don’t do anything we can’t. At least, this is what I think most people are thinking.

Games are just the opposite. Rather than games being an innate ability we have (like focusing your vision), you have an understanding of how and why you make decisions within a game. So when AI is making decisions about how a game is played, they are much more interesting because you could be in the same situation as the AI, making the same exact decisions. AI is popularly thought of as something that replaces human work. When AI plays a game, it feels closest to taking our position in the world, since games are things we have to consciously think about. Games that AI have played have thus captured the interest of news, here are the most important victories it has won.

1996–1997 Kasparov v. Deep Blue

On February 10th, 1996, IBM’s Deep Blue faced off against Gary Kasparov and became the first computer program to defeat a current chess world champion under normal conditions (i.e. with a clock in place). Unfortunately for Deep Blue, over the remaining five games in the match, it would lose three games and draw twice.

A second match was set for the next year, and Deep Blue’s team made upgrades to their software. On May 3rd Deep Blue opened with a loss. In the second game, Deep Blue had trouble calculating what to do, to prevent itself from wasting too much time it made a random move. The move was terrible, but this threw off Kasparov who assumed that Deep Blue could see further into the future of the game than he could. Rather than have a loss dragged out of him in front of millions of people, he resigned the second game (which is common in chess). In reality, Kasparov could have won the second game had he played it to completion, which was a small embarrassment to him, which led to him playing more recklessly in the following games, culminating in Deep Blue winning 2 games to Kasparov’s 1, tying 3.

As ESPN journalist Jeremy Schaap put it, “people were following this all over the world, people who had no interest in chess, people who only had interest in this narrative of man versus machine.” The interest in this game was fed by Kasparov’s claim at the end of the second game that IBM was cheating, which he based on the difference in Deep Blue’s quality of play between the first and second game. Chess has been a barometer of intelligence since its invention in the 6th century. To think that humans can be beaten by computers would be frustrating since there were still many ways which computers are inferior to humans. Of course, IBM was not cheating and Deep Blue’s quality change was attributed to its random move.

IBM’s public relations team had pumped up the event, and the company’s stock rose immensely. With a victory in the bag, IBM rejected a rematch, then dismantle Deep Blue and made its final resting place the Computer History Museum in Mountain View.

Deep Blue in the Computer History Museum

Kasparov no longer believes that Deep Blue was cheating, and recognizes that not only did Deep Blue beat him, but Deep Blue itself can be beaten by even new programs running on smaller computers. This game was a big step in the narrative of Artificial v. Human Intelligence and it established games as a method of comparison.

2011 IBM Watson Wins Jeopardy!

While eating lunch together, two IBM executives began commenting on the success of Ken Jennings on Jeopardy! who still holds the record for most consecutive games won. Perhaps in yearning for the success following Deep Blue’s victory, they created Watson, a question-answering system that was intended to beat Ken Jennings.

Watson, named after IBM’s founder Thomas J. Watson, was created in 2005, but did not begin its famous appearance until 2011. Along with Ken Jennings ,Watson went up against Brad Rutter, who were considered the two best Jeopardy! players at the time. Ken was a programmer who had taken a few AI classes and accepted the challenge partially because he thought there was no way an AI system could possibly beat him.

Ken Jennings, Watson, and Brad Rutter

Watson won the two matches it played against Brad and Ken, earning $77,147 to their $24,000 and $21,600, respectively. Watson then went on to defeat two members of congress, Rush D. Hold Jr. (a former Jeopardy! contestant) and Bill Cassidy, yet another bipartisan loss.

Unlike Deep Blue, Watson was spun off by IBM into many commercial products including Watson Text to Speech, Watson Speech to Text, Watson Natural Language Classifier, Watson Natural Language Understanding, Watson Assistant, and Watson Health, among others.

2013 DeepMind Beats Atari

While Watson was designed for a specific purpose, DeepMind Technologies wanted to create an AI system that need not be trained on a narrow field, moving us closer to the definition of General AI. The goal they set out on pursuing was beating the many Atari games with only one model that was not redesigned for any particular game.

DeepMind learns the best strategy for Breakout

Using only the input of what was on the screen and being told to maximize the score, DeepMind was able to beat every game at and get better scores then even the best human players. There 2013 paper, Playing Atari with Deep Reinforcement Learning led Google to acquire them in 2014.

2016 AlphaGo v. Lee Sedol

Deep Blue’s advantage had been its ability to process large amounts of moves at once and had a clever way of evaluating which one was the best. For the Asian board game Go, this was simply not possible. The rules of the game are quite simple, but their are around 2 * 10¹⁷⁰ number of states the game can be in at any one time. As a reference this number is larger than the number of atoms in the universe. Thus, strategy within a game of Go is hard to define since for each state, there are many decisions to evaluate that each lead to more states to evaluate. With no brute force approach working, DeepMind’s AlphaGo learned to attach value up to a few periods into the future, then learned to make decisions based on that value.

After defeating the European Go champion Fan Hui, AlphaGo was tasked with playing Lee Sedol, one of the highest ranking players ever. Beating Lee prompted him to retire, saying Even if I become the number one, there is an entity that cannot be defeated.”

2017 AlphaZero Masters Chess, Go, and Shogi

AlphaZero was a generalized version of AlphaGo, built with the intention of winning Chess, Go and Shogi (a Japanese version of chess). Not only did AlphaZero beat AlphaGo, it was done by only playing simulated games against itself, having no examples of expert’s games to look at. At the start of these simulations it knew absolutely nothing. It mastered Chess after 9 hours of training, Shogi after 2, and Go after 34.

While this was great for Google in attracting talent, AI professor Joanna Bryson has pointed out that the credit Google have been given for these achievements can give a massive negotiating advantage with governments seeking to regulate AI.

2019 AlphaStar

AlphaZero was then transitioned into AlphaStar with the intention of beating the real-time strategy game Starcraft. In 2019 AlphaStar achieved a ranking in the top 0.2 percent of human players. This was the first time that an AI had ever topped an e-sport. AlphaStar is not perfect at the Starcraft, but it is still seen as a massive milestone for AI, since it works in real-time, with partial information, under complex rules. While Go was relatively complicated in the number of decisions it faced, the rules to Go are quite simple compared to Srarcraft. This more complicated environment that AlphaStar operates in is a good indicator that AI is ready for wider commercial use like in self-driving vehicles, robotics, and supply chains.