What’s New in Deep Learning Research: Reducing Bias and Discrimination in Machine Learning Models…

Bias, unfairness and discrimination are permanent challenges in machine learning models. While humans can explain biased and discriminatory behaviors on the basics of cultural reasons, the biased decisions of machine learning algorithms remained a black-box most of the time. Bias is notoriously hard to quantify and remains hidden behind complex training datasets that contain systemic unfair and discriminatory decision-making patterns that, although imperceptible to human analysis, they slowly influence the behavior of algorithms. Recently, researchers from IBM introduced AI Fairness 360, an open source toolkit to detect discriminatory behavior in machine learning models.

Quantifying Fairness

Is this fair? That’s a question that has triggered centuries of studies in diverse fields such as psychology and economics. Encapsulating fairness in a single definition is almost impossible. Conceptually, fairness can be classified as “the absence of bias or discrimination on specific realms” but more specific definitions are very dependent on many facets such as domain, context and social constructs. For instance, computer scientists typically associate fairness with a property of unbounded nondeterminism while economists believe fairness is an attribute of the rational valuation of goods. Culturally, fairness is associated with behaviors that are not influence by person’s skin color, race or socio-economic status.

In the context of machine learning algorithms, there are three main perspectives that can influence the definition of fairness: Data vs. Mode, Group vs. Individual and WAE vs. WYSIWYG.

· Data vs. Model: Fairness can be measured at different stages in a machine learning pipeline. Specifically, fairness can be quantified in the training dataset or in the learned model of a machine learning solution.

· Individual vs. Group: Another friction that defines fairness in machine learning algorithms is Group vs. Individual. Group fairness, in its broadest sense, partitions a population into groups defined by protected attributes and seeks for some statistical measure to be equal across groups. Individual fairness, in its broadest sense, seeks for similar individuals to be treated similarly.

· WAE vs. WYSIWYG: We are all equal(WAE) defines fairness as an equal distribution of skills and opportunities among the participants in a machine learning task. The what you see is what you get(WYSIWYG) perspective reflects that the observations reflect ability with respect to the task. In a scenario that uses SAT scores as a feature for predicting success in college, the WYSIWYG worldview says that the score correlates well with future success and that there is a way to use the score to correctly compare the abilities of applicants. In contrast, the WAE worldview says that the SAT score may contain structural biases so its distribution being different across groups should not be mistaken for a difference in distribution in ability.

AI Fairness 360

AI Fairness 360 is a python library, a series of tools and examples that helps to identify and mitigate bias in a machine learning solution. AI Fairness 360 takes a holistic approach mitigating bias throughout the entire machine learning pipeline.

The initial release of AI Fairness 360 includes nine different algorithms that measure fairness across different stages of a machine learning workflow.

All algorithms are abstracted using a consistent programming model that can be easily injected into existing machine learning solutions.

Let’s see AI Fairness 360 in action in the context of a scenario that detects and mitigates bias in decisions to offer credit in Germany. The goal of the model would be to determine if an applicant should be given credit based on various features from a typical credit application. The protected attribute will be “Age”, with “1” and “0” being the values for the privileged and unprivileged groups, respectively.

The first step to use AI Fairness 360 would be to install the library as a typical Python package.

pip install aif360

As our first step, we are going to load a training dataset based on historical credit decisions in Germany and setup the protected attribute.

dataset_orig = load_preproc_data_german(['age'])

dataset_orig_train, dataset_orig_test = dataset_orig.split([0.7], shuffle=True)

privileged_groups = [{'age': 1}]
unprivileged_groups = [{'age': 0}]

At this moment, we can compute fairness in the training dataset. An easy way to do this would be to compare the percentage of favorable results for the privileged and unprivileged groups, subtracting the former percentage from the latter. A negative value indicates less favorable outcomes for the unprivileged groups. This is implemented in the method called mean difference on the BinaryLabelDatasetMetric class as shown in the following code:

metric_orig_train = BinaryLabelDatasetMetric(dataset_orig_train, 

AI Fairness 360 provides a series of tools that clearly illustrates clear bias in 4 out of 5 key metrics.

At this point, we can choose several algorithms to mitigate the impact of the biased training dataset. In this case, we will choose the Optimized Preprocess algorithm, which is implemented in “OptimPreproc” class in the “aif360.algorithms.preprocessing” directory. This algorithm will transform the dataset to have more equity in positive outcomes on the protected attribute for the privileged and unprivileged groups. We will recompute the bias indicators using the same method applied to the training dataset.

optim_options = {
"distortion_fun": get_distortion_german,
"epsilon": 0.01,
"clist": [0.99, 1.99, 2.99],
"dlist": [.1, 0.05, 0]

OP = OptimPreproc(OptTools, optim_options,
unprivileged_groups = unprivileged_groups,
privileged_groups = privileged_groups)

OP = OP.fit(dataset_orig_train)
dataset_transf_train = OP.transform(dataset_orig_train, transform_Y = True)

dataset_transf_train = dataset_orig_train.align_datasets(dataset_transf_train)
metric_transf_train = BinaryLabelDatasetMetric(dataset_transf_train, 

The results clearly showed a decreased on the bias indicators of the algorithm.

Building fairness and non-discriminatory behavior into AI models is not only a matter of technological advantage but of social responsibility. Efforts like AI Fairness 360 are an important step in the right direction and, hopefully, we will soon see fairness optimization algorithms included in some of the most popular machine learning frameworks.

Source: Deep Learning on Medium