The AI Ecosystem is a MESS

Original article can be found here (source): Artificial Intelligence on Medium

The AI Ecosystem is a MESS

Why is it impossible to understand what AI companies really do?

Traffic Chaos By sladkozaponi via Shutterstock

The State of AI in Businesses

Over the last several years, there’s been a rush to find out how to integrate AI into businesses, and it’s no secret that doing so could offer huge comparative advantages. But for all the hype, AI in businesses is still very much in the early phase.

Our team hails from Uber, Google, Facebook and Adobe where we’ve seen both the positives and challenges of deploying AI across business lines. Most companies don’t have the same resources to build in-house tools, deeply measure results and fund extensive research. Our goal with this blog is to use our in-depth knowledge of the AI space to make sense of the ecosystem, cut through the hype, and provide insights that can help you with your AI investment decisions across the pipeline.

In-House Data Science versus External Partner

Many companies are leveraging AI to upend areas of business by applying models to common business problems in various business verticals. We consider these companies as vertical business solutions — they sell software that is “powered” by AI. They target various teams such as customer service, marketing, finance and ops with a SAAS solution to solve their problems. The other set of companies are those selling software solutions to empower your Data Science, Data Engineering and ML engineering teams.

The rest of this blog is focused on companies in the latter space, AI/ML Infrastructure companies that are building software to help your teams build and deploy models themselves.

Complexity of the AI Space

One of the first problems we hear from enterprise teams is that it’s extremely hard to figure out the right problems to solve with AI. Even when they find the right problems, we commonly hear that most of the models that they develop never make it to production. It’s such a common problem that we know of a company building a model to predict whether other models will actually make it to production. You have to wonder if that model itself will make it into production.

At various Data Science team sizes and scaling points, you find that there are new unique complexities that arise out of the Data Science organization. Some of the complexities are really software engineering problems that have taken on a new form and others are unique to this space. All of this is compounded by the confusing suite of software tools and solutions that are built to “help” Data Science. Enter stage left, where you have a 100+ companies who are pitching confusing software solutions to “Empower your Data Science teams”.

How to Evaluate ML/AI Software Companies

AI/ML Infrastructure tools is an extremely crowded space. I can point to 100+ companies offering services in hyper-parameter tuning, deploying, MLOps, governance, control, auditing, regulation, explainability, performance tuning, you name it. Even as a technical person, I’ve been pitched by some ML tools companies, where after an hour pitch, I still don’t know what they do.

The nebulous quotes abound include “Achieve AI scale”, “Optimize AI”, “Empower AI”, “Take control of your AI” — but what do these software solutions actually do?

There is a simple mental model we use to section the space, place the solutions, and cut through the hype.

ML Workflow Stages Diagram by Author

Production or Pre-Production?

As you look at the ML/AI Infrastructure space, the first segmentation of products to make is pre-production or production.

Pre-Production

  • Is the Software helping you build a better model, auto-choose the right model, make the model build process reproducible, audit model builds, produce better data for model builds, or track training runs?

Production

  • Is the Software helping you integrate your model into your business, scale your model into your product, troubleshoot problems in production, explain results for the rest of your team, or provide A/B effective analysis of models?

The reason this segmentation matters is that as a Data Science team grows, different people in the organization are eventually responsible for production versus pre-production. Software solutions built for one function/organization don’t serve all people well across the organization.

Model Workflow Stages

The next segmentation of products to make is to where in the ML workflow does the product fit. Broadly, we see three main stages — Data Prep, Model Building, and Production.

Data Preparation involves preparing, labeling, and cleaning the data to be used for models. It’s where the lion’s share of time and energy goes in terms of using AI models, and it’s primarily under the domain of data engineers and data wranglers.

Model building involves feature selection, training pipelines, hyper-parameter tuning, explainability analysis, pre-production audits and evaluation. It is usually taken on by data scientists.

Finally, there’s production, which involves deploying models for inference (generating predictions), performance monitoring, troubleshooting and production based explainability. It’s usually in the machine learning engineer wheelhouse.

Each of these broad categories of the ML Workflow (Data Prep, Model Building, and Production) has a deeper set of features. Here’s a more detailed overview of the ML Workflow stages with product capabilities needed in each stage and companies offering features in those verticals today.