Simulation is everywhere!

Original article was published by Allison Liemhetcharat on Artificial Intelligence on Medium

Simulation is everywhere!

Welcome to my second article on simulation. If you’d like to read my first article on what simulation is, click here! This article focuses on the different domains that simulation has been used in.

tl;dr of this article: Simulation can and has been used in a wide variety of domains.

In my previous article, we learned how simulation is about making something “similar enough”, and we primarily considered two domains: video games and professional training. Besides these two domains, simulations have been used in many others too, and we will explore more of them in this article.

Sony Aibo ERS-7, a simulation of a dog. Credit: Sony.

Below is a video of starling murmuration, which is when thousands of starlings (a kind of bird) fly together in dazzling patterns. The video below is of real starlings flying in the real world, so these are not simulations 😃

Why am I showing a video of real birds in an article about simulation? Well, for two reasons. First, starling mumurations are mesmerizing and beautiful. And second, I’m not the only one who thinks so. Other people in artificial intelligence were also interested in such flocking behavior, and sought to figure out how to replicate it on the computer.

One of the first, and possibly the most well-known, flocking algorithms is the Boids model by Craig Reynolds in 1986. Essentially, with 3 basic rules on each agent (or simulated bird, in this case), an overall flocking behavior emerges. The 3 rules are: separation (so the agents don’t get too close to one another), steering (so agents head in similar directions), and cohesion (so agents don’t get too far from one another). The video below demonstrates 2D flocking using these rules, with an additional rule to avoid obstacles in general).

Nowadays, flocking algorithms are much more commonplace, and are even available as part of Adobe After Effects.

Perhaps you’re wondering if you’ve seen flocks in animations and shows? Flocking has been simulated in a large variety of shows. For example, the following scene from The Lion King is one of the most emotional parts of the show, and it also demonstrates flocking.

Flocking was probably used in the birds in the sky at the beginning of the stampede, but more importantly, the wildebeests themselves were simulated with a flocking algorithm. For more details, check out The Lion King Film Notes (search for the second instance of “stampede”).

Thus, simulation has been used in graphics and animation! Going back to the Boid algorithm, each “agent” is simulated independently with the 3 rules. This concept of independent agents exhibiting complex behaviors is frequently used in multi-agent and multi-robot research, and unsurprisingly, simulation is also widely used there.

I’ve been involved in RoboCup for a number of years during my undergraduate and PhD studies at Carnegie Mellon University. The ultimate goal of RoboCup is: “By the middle of the 21st century, a team of fully autonomous humanoid robot soccer players shall win a soccer game, complying with the official rules of FIFA, against the winner of the most recent World Cup.

There are many leagues in RoboCup, and I participated in soccer leagues (the Standard Platform League and the 4-legged league before that), and served on the Organizing and Technical Committees of the RoboCup Standard Platform League. The soccer leagues focus on robots playing soccer as a team against an opposing team, where the robots act autonomously, i.e., no human input. Below is a video of a RoboCup Soccer 2D Simulation game.

In the 2D Simulation League, teams play 11 vs 11 (identical to human teams). The simulation is 2D, and the entire field is simulated to be sized similarly to a “real” soccer field. The simulated robots dribble and pass the ball in order to score goals, similar to human players.

RoboCup also has a 3D Simulation League, where 3D simulated robots play soccer against one another. The key difference between the two leagues is that the 3D Simulation League simulates 3 dimensions, while the 2D Simulation League simulates 2 dimensions, as their names suggest.

In particular, the environment in the 3D Simulation League is simulated to have 3D physics, and thus the robots are required to maintain their balance as they walk on the field and kick the ball (not as easy a feat as it may sound).

Let’s go one step further, to the Standard Platform League (SPL) that I participated in. In the SPL, two teams of physical robots play soccer against each other.

You may notice similarities between the real robots in the SPL and the simulated robots in the 3D Simulation League. Both use the Nao Humanoid Robots, although one is in the real world and one is simulated. However, behaviors learned in simulated can and has been applied to the real robots.

In addition, the real robots themselves are simulations of human players, because ultimately, the research in RoboCup is to create realistic robot soccer players! Hence, simulation has been widely used in scientific research, in particular for robotics.

Besides robotics research, simulation has also been used for robotics in general. For example, in the video below, simulation is used to improve the process of manufacturing cars.

We’ve discussed the simulations of agents/robots, and simulations of cars (or at least the manufacturing of them). Putting the two together brings us to another domain of my interests: simulating cities and traffic!

One my favorite video games is Cities: Skylines. In this game, you are the mayor of a city, and build roads, designate residential, commercial, and industrial areas, and build facilities such as schools, hospitals etc. It’s similar to games like SimCity, and ultimately, such games simulate the development of a city, its traffic and people.

Cities: Skylines is a video game that I enjoy, and we’ve previously discussed simulation in video games. In addition, Cities: Skylines does have a somewhat realistic simulation of traffic in a city, that changes as you develop the city and design the road networks.

Next, we have traffic simulators, such as Aimsun and PTV Vissim. The purpose of traffic simulators is to provide realistic simulations of traffic, to help real-world city planners and governments to plan new roads and investigate impacts of new roadways and traffic policies.

When comparing the video above of Aimsun with the video of Cities: Skylines, the latter has better graphics, while the former is much more realistic. Which is the “better” simulation? It depends on the application! Aimsun is arguably better for city planners, while Cities: Skylines is better for gamers. Regardless of the use case, simulation is used by both 😄

If we combine the domains of robots and traffic, we arrive at a topic near and dear to my heart: self-driving vehicles! I won’t be going into the details of how self-driving vehicles work, and there are many articles that you can read about it.

Let’s first consider simulating vehicles in general. Arguably, the simulated vehicles in video games such as Cities: Skylines, and traffic simulators such as Aimsun are “self-driving” within their simulated environments. However, in many such situations, the vehicles have perfect global knowledge of the environment, i.e., they perfectly know what and where everything is. What would it be like to drive a vehicle in a simulated environment, with imperfect information? Below are some examples!

As before, we shall start with a video game! Above is a gameplay video of Grand Theft Auto V. The game itself is mired in controversy, but its simulation of cities and vehicles is realistic enough that some self-driving vehicle R&D wanted to use it as a simulator to train the self-driving algorithms.

When people play games such as Grand Theft Auto, they are driving a simulated vehicle in a simulated environment. Vehicle simulations are also used for professional training, either in a completely virtual environment, or a mixed one, like the video below.

Here we have a physical bus overlaid with screens to simulate the driving experience. Bus simulators are used to train people on how to drive buses on the road, which is much safer than training on the road.

Let’s now consider the simulation of self-driving vehicles! One common approach is to have a simulated vehicle in a completely virtual environment, where all the sensor inputs to the vehicle are simulated, and the outputs of the self-driving software act in the virtual environment. Here is a video using Udacity’s Self-Driving Car Simulator, in order to train a self-driving algorithm with convolutional neural networks (CNNs).

Self-driving vehicle simulation doesn’t always have to be a virtual vehicle in a virtual environment. Self-driving vehicle simulation can also be done with a real vehicle in a real environment!

The video above shows Waymo’s self-driving vehicle running in their test track. The situation above is a simulation because ultimately, they are simulating real-world situations, such as another driver cutting the vehicle off at a four-way stop intersection. This simulation seeks to replicate possible scenarios in the real-world that may be difficult or dangerous to test in, and thus it is safer to do so in a repeatable, simulated environment.

In conclusion, simulation can and has been used in a variety of domains, ranging from video games, professional training, animation, research, robotics, city planning, and self-driving vehicles. There are many more domains where simulation is also being used but are not covered here, thus showing how pervasive simulation is in our daily lives!