Tackling Bias: AI Needs Equity, Not Equality

Original article was published by Nilan J. Lovelace on Artificial Intelligence on Medium


There are two things that most people will say cannot possibly be bias: data and themselves. In a way, they are right. Neither data nor any single individual are inherently bias. While neither data nor people are inherently bias, that doesn’t mean they can’t be influenced by and indoctrinated into existing biases that stick with them for life.

In a previous post I asked the question “Can Machine Learning Eliminate Racial Disparities in Medicine?” Many companies applying machine learning and artificial intelligence to the medical field, that answer would be a resounding “Yes!” If you dig a little deeper, you might get replies like “Our algorithms don’t take race into consideration” or “We’ve taken a standardized approach to accessing patient health to limit preconceived biases in how they’re treated.” Yet, Black and non-White Latino patients have found that the disparaties have remained the same, if not worse, after algorithms, meant to manage care for nearly 200 Million people per year, had been implemented. If these algorithms were made to be colorblind, why are Black and non-White Latino patients not receiving the same quality of care?

In March of 2019, a study showed that imaging systems for self-driving cars are 5% less likely to detect darker skin, under any light condition, than it would lighter skin. Researchers initially recognized this problem after noticing that Black people made up an unusually high percentage of incidents involving self-driving vehicles. Though the models responsible for self-driving cars weren’t built with bias in mind, a bias system was still built, not only from the bias of the modelers, but also the data.

The problem with a colorblind approach is that there’s an assumption that there is no bias because we didn’t build it to be bias. Since we didn’t specifically model the algorithm to produce inequitable outputs based on race, it couldn’t possibly be inequitable. By building a model that made decisions based on data and standard practice alone, we eliminated any chance for bias to be introduced after the algorithm is live. But what would happen if, instead of having bias introduced, the bias had always been there?

Biases and blind spots exist in big data as much as they do in individual perceptions and experiences. Yet there is a problematic belief that bigger data is always better data and that correlation is as good as causation.

— Kate Crawford

Colorblind Systems aren’t Colorblind

Colorblind approaches are only effective if you are proactively against being colorblind. These are only two examples of real world consequences that come from colorblinds systems not considering that even raw data can be bias. By ignoring race, a healthcare system never considers that Black patients spend the same total amount on health care as White patients because of repeated low cost, low quality options and higher rates of illness. That oversight has caused only 17% of Black patients to receive adequate referrals oppose to the 45% of Black patients who should have received additional care.

By ignoring race, engineers never take into consideration that color film, an extremely important artifact of computer vision, was specifically made to capture only white skin, which in turn makes all modern imaging systems negatively biased towards dark skin. Not only does this lead to Black pedestrians being more likely to be hit by a Tesla, it also means that facial recognition software is less efficient, like when Apple’s FaceID could not distinguish between face for some Asian and Black users or the high false positive rates for people of color in Police-operated software.

If we want to build smart, equitable systems, we need to be proactive in building smart, equitable solutions to existing biases that have probably existed far longer than many of us have been alive. As data scientist, data engineers, and machine learning modelers, it’s our responsibility not only to create ethical systems that work, but systems work for us all.