Unboxing the “Black Box” Models

Original article was published on Artificial Intelligence on Medium

Model Agnostic Interpretable Methods:

Feature Importance:

  • Permutation Feature Importance
  • Feature Interaction

Causal Interpretation:

  • Partial Dependence Plots (PDP)
  • Individual Conditional Expectation (ICE)

Surrogate Models:

  • Global Surrogate
  • Local Surrogate — LIME

Explain Predictions:

In this article, we will be using these model agnostic methods to interpret the results of Gradient Boosting Regressor model we created in the last article.

Feature Importance:

When business asks questions like “Why did our customers churn out?” or “What leads to more customer retention?”, it is important to understand the features that affect these prediction.

Permutation Feature Importance:

Permutation feature importance measures the increase in the prediction error of the model after we permuted the feature’s values, which breaks the relationship between the feature and the true outcome.

So if a feature is “important”, shuffling its values increases the model error.

Python Implementation — ELI5

Permutation Feature Importance using ELI5

Higher weight indicates higher feature importance.

Features like hour, working day, temp, humidity are important and hence changing value drastically changes the model predictions.

Feature Interaction:

Feature Interaction measures the variance in Partial dependence function of one feature i.e. how does permutation in one feature value affects the other feature. If the variance is high, then the features interact with each other, if it is zero, they do not interact.

Lets look into Correlation matrix to find out features that are highly correlated and then look into their feature interaction.

Correlation between important feature

Features humidity — windspeed and temperature — humidity have negative correlation. Lets look into their feature interaction now.

Theory: Friedman’s H-statistic:

According to Friedman’s H-statistic we deal with 2 cases:

  • 2 way interaction measure that tells us to whether and to what extend the two features interact with each other
  • Second, total interaction measure to tell us whether and to what extend a feature interacts with all other features in a model

Python Implementation — H-statistics

H statistic of the variables represented by the elements of array_or_frame and specified by indices_or_columns can be computed. The larger H, the stronger the evidence for an interaction among the variables. H varies from 0 to 1.

{('temp', 'atemp'): 0.15373552315496558,
('temp', 'humidity'): 0.09849995273815548,
('temp', 'windspeed'): 0.5574920397015759,
('atemp', 'humidity'): nan,
('atemp', 'windspeed'): nan,
('humidity', 'windspeed'): 0.4392361526105014}

Interaction between Temp & Windspeed and Humidity & Windspeed are really high.