自定义 CNN 模型问题上 SHAP 值的可解释性

Question

我正在尝试使用 SHAP 解释训练模型的预测。我遵循以下源代码，在 Imagenet 数据集中使用 RESNET50 可以正常工作

from tensorflow.keras.applications.resnet50 import ResNet50, 
preprocess_input

import json
import shap
import tensorflow as tf

# load pre-trained model and choose two images to explain
model = ResNet50(weights='imagenet')
def f(X):
    tmp = X.copy()
    print(tmp.shape)
    input()
    preprocess_input(tmp)
    return model(tmp)
X, y = shap.datasets.imagenet50()


# load the ImageNet class names as a vectorized mapping function from ids to names
url = "https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json"
with open(shap.datasets.cache(url)) as file:
    class_names = [v[1] for v in json.load(file).values()]
    
print(len(class_names))
print(X.shape)   
input()    

# define a masker that is used to mask out partitions of the input image, this one uses a blurred background
masker = shap.maskers.Image("inpaint_telea", X[0].shape)

# By default the Partition explainer is used for all  partition explainer
explainer = shap.Explainer(f, masker, output_names=class_names)

# here we use 500 evaluations of the underlying model to estimate the SHAP values
shap_values = explainer(X[1:3], max_evals=500, batch_size=50, outputs=shap.Explanation.argsort.flip[:1])
shap.image_plot(shap_values)

这是可以找到的确切示例HERE

现在我有另一个 RESNET50，但接受了另一个考虑 12 个类的多类分类的训练。我改编了上面的源代码，但运行时出现问题。

import numpy as np
import tensorflow.keras.models import load_model, preprocess_input
import shap

def f(data_to_explain):
    tmp = data_to_explain.copy()
    preprocess_input(tmp)
    return model(tmp)

reconstructed_model = load_model("my_model")
data_to_explain=np.load("data_to_use.npy")
class_names = ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11"]

# define a masker that is used to mask out partitions of the input image, this one uses a blurred background
masker = shap.maskers.Image("inpaint_telea", data_to_explain[0].shape)

explainer = shap.Explainer(f, masker, output_names=class_names)

# here we use 500 evaluations of the underlying model to estimate the SHAP values
shap_values = explainer(data_to_explain[1:3], max_evals=500, batch_size=50, outputs=shap.Explanation.argsort.flip[:1])
shap.image_plot(shap_values)

我收到的错误是：

 shap_values = explainer(data_to_explain[1:3], max_evals=500, batch_size=50, outputs=shap.Explanation.argsort.flip[:1])
  File "/usr/local/lib/python3.8/dist-packages/shap/explainers/_partition.py", line 135, in __call__
    return super().__call__(
  File "/usr/local/lib/python3.8/dist-packages/shap/explainers/_explainer.py", line 310, in __call__
    sliced_labels = [labels[index_list] for index_list in output_indices]
  File "/usr/local/lib/python3.8/dist-packages/shap/explainers/_explainer.py", line 310, in <listcomp>
    sliced_labels = [labels[index_list] for index_list in output_indices]
IndexError: index 852 is out of bounds for axis 0 with size 12

代码与上一个基本相同，所有形状都匹配，唯一的区别是类的数量。那么，我的问题可能是什么？

P.s=你可以重现我的错误HERE

Answer 1

#Try it
for i in range(len(data_to_explain)):
    shap_values = explainer(np.asarray([data_to_explain[i]]), max_evals=500, outputs=shap.Explanation.argsort.flip[:1])
    shap.image_plot(shap_values)