Original article was published on Deep Learning on Medium
Seven Key Dimensions to Understand Any Machine Learning Problem
Number of agents involved, nature of the observations, stochasticity, dynamism, are some of the characteristics that will help us understand a machine learning problem.
Tackling a machine learning problem might feel overwhelming at first. What model to choose?, which architecture might work best? In a process that is mostly driven by trial and error experimentation, those decisions result incredibly important. One aspect that really helps to navigate that universe of decisions is to clearly understand the nature of the problem. In machine learning scenarios, an important part of understanding the problem is based on understanding its environment.
Every machine learning problem is a new universe of complexities and unique challenges. Very often, the most challenging aspects of solving an AI problem is not about finding a solution but understanding the problem itself. As paradoxically as that sounds, even the most experienced AI experts have been guilty of rushing into proposing deep learning algorithms and exotic optimization techniques without fully understanding the problem at hand. When we think about a machine learning problem, we tend to link our reasoning to two main aspects: datasets and models. However, that reasoning is ignoring what can be considered the most challenging aspect of an AI problem: the environment.
When designing machine learning solutions, we spend a lot of time focusing on aspects such as the structure of learning algorithms [ex: supervised, unsupervised, semi-supervised], the architecture of a neural network [ex: convolutional, recurrent…] or the characteristics of the data [ex: labeled, unlabeled…]. However, little attention is often provided to the nature of the environment on which the machine learning solution operates. As it turns out, the characteristics of the environment are the number one element that can make or break an machine learning model.
There are several aspects that distinguish machine learning environments. The shape and frequency of the data, the nature of the problem , the volume of knowledge available at any given time are some of the elements that differentiate one type of machine learning environment from another. Deep diving into those characteristics will guide the strategies of machine learning experts in areas such as algorithm selections, neural network architectures, optimization techniques and many other relevant aspects of the lifecycle of machine learning applications. Understanding a machine learning environment is an incredibly complex task but there are several key dimensions that provide clarity on that reasoning.
Seven Key Dimensions to Classify an AI Environment
One of the most effective methodologies for understanding a machine learning environment is to classify it across a series of well-known dimensions that are often only segmented in two or three classifications. Among the different characteristics that can be used to classify a machine learning environment, there are seven key exclusive dynamics that provide a rapid understanding of the challenges and capabilities needed by AI agents.
1-Single Agent vs. Multi-Agent
One of the most obvious dimensions to classify a machine learning environment is based on the number of agents involved. The vast majority of machine learning models today focus on environments involving a single agent but there is an increasing expansion in multi-agent settings. The introduction of multiple agents in a machine learning problem raises challenges such as collaborative or competitive dynamics which are not present in single-agent environments.
2-Complete vs. Incomplete
Complete AI environments are those on which, at any given time, the agents have enough information to complete a branch of the problem. Chess is a classic example of a complete machine learning environment. Poker, on the other hand, is an incomplete environments a machine learning strategies can only anticipate many moves in advance and, instead, they focus on finding a good ‘equilibrium” at any given time. Most of the famous Nash equilibrium principles are particularly relevant in incomplete machine learning environments.
3-Fully Observable vs. Partially Observable
A fully observable machine learning environment has access to all required information to complete target task. Image recognition operates in fully observable domains. Partially observable environments such as the ones encountered in self-driving vehicle scenarios deal with partial information in order to solve machine learning problems. Partially observable environments often rely on statistic techniques to extrapolate knowledge of the environment.
4-Competitive vs. Collaborative
Competitive machine learning environments face agents against each other in order to optimize a specific outcome. Games such as GO or Chess are examples of competitive machine learning environments. Collaborative machine learning environments rely on the cooperation between multiple machine learning agents. Self-driving vehicles or cooperating to avoid collisions or smart home sensors interactions are examples of collaborative machine learning environments. Many multi-agent environments such as video games include both collaborative and competitive dynamics which makes them particularly challenging from an AI perspective.
5-Static vs. Dynamic
Static AI environments rely on data-knowledge sources that don’t change frequently over time. Speech analysis is a problem that operates on static machine learning environments. Contrasting with that model, dynamic machine learning environments such as the vision machine learning systems in drones deal with data sources that change quite frequently. Dynamic machine learning environments often need to enable faster and more regular training of machine learning agents.
6-Discrete vs. Continuous
Discrete machine learning environments are those on which a finite [although arbitrarily large] set of possibilities can drive the final outcome of the task. Chess is also classified as a discrete AI problem. Continuous machine learning environments rely on unknown and rapidly changing data sources. Multi-player video games are a classic example of continuous machine learning environments.
7-Deterministic vs. Stochastic
Deterministic machine learning environments are those on which the outcome can be determined base on a specific state. By determinism, we specifically refer to machine learning environments that ignore uncertainty. Most real world machine learning environments are not deterministic. Instead, they can be classified as stochastic. Self-driving vehicles are a one of the most extreme examples of stochastic machine learning environments but simpler settings can be found in simulation environments or even speech analysis models.
Understanding a machine learning environment is one of the most challenging steps on any machine learning problem. Luckily, the friction points across the seven dimensions explored in this article often yield a robust classification of an AI environment and facilitate the selection of models and architectures. While there have been notorious advancements in machine learning architectures and optimization techniques, the analysis of environments remains a highly subjective aspect of the AI lifecycle.