SHAP Values for Image Classification Tasks (Keras)¶

Deep learning models like convolutional neural networks are giving quite good results at many computer vision tasks. We need to understand that the models that are giving such high accuracy are predicting results based on data parts that they should use for prediction. Let's say for example that we have an image classification task of predicting cat vs dog then the model should look at pixels of face and body of cat/dog to predict class, not the background pixels of images should be used to make a decision. If that is the case then we can be sure that our model has generalized better and actually learning features of cats and dogs. There are many prediction interpretation libraries but as a part of this tutorial, we'll be using SHAP. SHAP is a python library that generates shap values for predictions using a game-theoretic approach. We can then visualize these shap values using various visualizations to understand which features contributed to prediction. We have a starter tutorial on SHAP where we discuss how to use it for tabular (structured) datasets. Please check the below link if you want to refer to it.

• SHAP - Explain Machine Learning Model Predictions using Game-Theoretic Approach

As a part of this tutorial, we have designed a simple CNN using keras and trained it with the Fashion MNIST dataset. Then, we have explained correct and incorrect predictions using SHAP python library.

Below, we have listed important sections of tutorial to give an overview of the material covered.

Important Sections Of Tutorial¶

1. Load Data
2. Define CNN
3. Compile And Train Model
4. Evaluate Model Performance
5. Explain Predictions Using SHAP Partition Explainer
   • Visualize SHAP Values For Correct Predictions
     • Image Plots
   • Visualize SHAP Values For Incorrect Predictions
6. Explain Predictions Using SHAP Permutation Explainer
   • Visualize SHAP Values For Correct Predictions
   • Visualize SHAP Values For Incorrect Predictions

Below, we have imported the necessary libraries and printed the versions that we have used in our tutorial.

1. Load Data ¶

In this section, we have loaded the Fashion MNIST dataset available from keras. The dataset has grayscale images of shape (28,28) pixels for 10 different fashion items. The dataset is already divided into the train (60k images) and test (10k images) sets. Below we have included mapping from index to fashion item names.

2. Define CNN ¶

In this section, we have defined a convolutional neural network that we'll use to classify images of the Fashion MNIST dataset loaded earlier. The network is simple with 2 convolution layers and one dense layer. The two convolution layers have filters 32 and 16 respectively and both are using kernel size of (3,3). We have applied relu (rectified linear unit) activation after both convolution layers. The output of the second convolution layer is flattened and then given to the dense layer. The dense layer has 10 output units, the same as the number of target classes.

3. Compile And Train Model ¶

In this section, we have first compiled our model to use Adam optimizer, cross entropy loss, and accuracy metric. Then, we have trained the model for 10 epochs using train and validation data. We can notice from the results getting printed after each epoch that the model has achieved near 91% accuracy on validation data and 93% accuracy on train data. It seems to be good accuracy and we can now evaluate the predictions made by our model.

4. Evaluate Model Performance ¶

In this section, we have evaluated the performance of our network by calculating accuracy, confusion matrix, and classification report (precision, recall, and f1-score per class) metrics. We can notice from the results that our model is doing good for categories except 'shirt', 'coat', 'pullover' and 'top' which are almost the same hence can be confusing for the model to differentiate between them in grayscale images. Many 'shirt' category images are confused with 't-shirt/top' category as there is quite less difference between them.

We have used the function available from scikit-learn to calculate various metrics. If you want to learn about various ML metrics available from sklearn then please feel free to check the below link which covers the majority of them.

• Scikit-Learn - Model Evaluation & Scoring Metrics

5. SHAP Partition Explainer ¶

In this section, we have used SHAP partition explainer to explain the predictions made by our network. We have tried to explain both correct and wrong predictions of our model to see which parts of an image are contributing to predictions.

In order to use SHAP, we first need to initialize it by calling initjs() function.

Then, we have created an instance of Partition explainer by using Explainer() constructor. We have given three values to the constructor.

• Our trained model
• Masker to mask parts of an image using blurring or inpainting.
• List of Target Class labels

The Explainer() constructor by default creates an instance of Partition explainer. We can generate shap values for images using this explainer object.

Visualize SHAP Values For Correct Predictions¶

In this section, we have explained the correct predictions made by our model. We have first taken 4 sample images from our test dataset and generated shap values for them using the partition explainer object we created earlier. Then, we have printed the actual labels, predicted labels, and the model's predicted probabilities.

Image Plot¶

In this section, we have generated image plots that visualizes shap values generated by the explainer object.

Below, we have generated the first image plot using shap values generated in previous cells. The chart shows the actual image and parts of it highlighted in shades of red and blue colors. The shades of red color show parts that contributed positively and shades of blue color show parts that contributed negatively to the prediction of that category. It also shows the first five categories that the model thinks the image belongs to.

Below, we have generated another image plot using a different masker named inpaint_ns. We have created masker and explainer instances for this again.

Below, we have generated another image plot using masker that uses blurring-based masker. We can notice that the blurring masker seems to be doing a good job compared to other maskers. The tuple of integer values that we provide in the string of masker is the size of the kernel used to blur.

Visualize SHAP Values For Incorrect Predictions¶

In this section, we have explained wrong predictions using the explainer object. In order to explain wrong predictions, we have first retrieved indexes of all wrong predictions from the test set. Then, we have used indexes of wrong predictions to retrieve those samples and make predictions on them again to retrieve the probabilities of the model for those predictions.

Below, we have generated an image plot using by generating shap values for wrong predictions. In the next cell, we have explained that we can generate visualization using image() function of plots sub-module of shap library.

6. SHAP Permutation Explainer ¶

In this section, we are trying another explainer available from a shap named permutation explainer. The permutation explainer can be created using PermutationExplainer() constructor and accepts the same parameters as the permutation explainer. The permutation explainer tries different combinations of features to generate shap values.

Below, we have first created a permutation explainer using model and masker objects.

Visualize SHAP Values For Correct Predictions¶

In this section, we have generated shap values for 4 test images using the permutation explainer object. In the next cell, we have also printed actual labels, predicted labels, and probabilities of those 4 sample images. We have also calculated labels according to the 10 probabilities generated by our model.

In this below cell, we have plotted an image plot showing shap values that contributed to predictions.

Visualize SHAP Values For Incorrect Predictions¶

In this section, we have generated shap values for wrong predictions. The majority of code in this section is a repeat of earlier sections hence we have not included repeated explanations for them.

This ends our small tutorial explaining how we can generate SHAP values for image classification networks created using keras to explain predictions made by the model. Please feel free to let us know your views in the comments section.

References¶

• SHAP - Explain Machine Learning Model Predictions using Game Theoretic Approach
• Scikit-Learn - Model Evaluation & Scoring Metrics
• Convolutional Neural Networks (CNN or ConvNet)