Keras后端平均功能：“'float'对象没有属性'dtype'”？

Question

我正在尝试为使用 Keras 的网络引入新的内核正则化。但是，它给了我错误： 'float' 对象没有属性 'dtype' 我该如何解决？

我在这里找到了代码：KL divergence in keras(tensorflow backend)

这是我的代码：

    from keras import backend as K

    kullback_leibler_divergence = keras.losses.kullback_leibler_divergence

    def kl_divergence_regularizer(inputs):
        means = K.mean((inputs))
        return 0.5 *(0.01 * (kullback_leibler_divergence(0.05, means)
                      + kullback_leibler_divergence(1 - 0.05, 1 - means)))


    model = Sequential([
        Dense(100, input_shape=(x_train_s.shape[1],),kernel_initializer='random_uniform'),
        Activation('elu'),
        Dense(x_train_s.shape[1],kernel_initializer='random_uniform', kernel_regularizer=kl_divergence_regularizer),
        Activation('tanh')
    ])

    model.compile(optimizer='adam',loss='mean_squared_error')

    model.fit(x_train_s, x_train_s, epochs=5,validation_split=0.1, shuffle=True, verbose=1,batch_size=np.uint(x_train_s.shape[0]/100))

此处显示了完整的错误追溯：

AttributeError                            Traceback (most recent call last)
<ipython-input-39-59bc90c687de> in <module>
     39     Activation('elu'),
     40     Dense(x_train_s.shape[1],kernel_initializer='random_uniform', kernel_regularizer=kl_divergence_regularizer),
---> 41     Activation('tanh')
     42 ])
     43 

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\sequential.py in __init__(self, layers, name)
     91         if layers:
     92             for layer in layers:
---> 93                 self.add(layer)
     94 
     95     @property

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\sequential.py in add(self, layer)
    179                 self.inputs = network.get_source_inputs(self.outputs[0])
    180         elif self.outputs:
--> 181             output_tensor = layer(self.outputs[0])
    182             if isinstance(output_tensor, list):
    183                 raise TypeError('All layers in a Sequential model '

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\base_layer.py in __call__(self, inputs, **kwargs)
    429                                          'You can build it manually via: '
    430                                          '`layer.build(batch_input_shape)`')
--> 431                 self.build(unpack_singleton(input_shapes))
    432                 self.built = True
    433 

C:\ProgramData\Anaconda3\lib\site-packages\keras\layers\core.py in build(self, input_shape)
    864                                       name='kernel',
    865                                       regularizer=self.kernel_regularizer,
--> 866                                       constraint=self.kernel_constraint)
    867         if self.use_bias:
    868             self.bias = self.add_weight(shape=(self.units,),

C:\ProgramData\Anaconda3\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint)
    253         if regularizer is not None:
    254             with K.name_scope('weight_regularizer'):
--> 255                 self.add_loss(regularizer(weight))
    256         if trainable:
    257             self._trainable_weights.append(weight)

<ipython-input-39-59bc90c687de> in kl_divergence_regularizer(inputs)
     14     means = K.mean((inputs))
     15 #     means=1e-6
---> 16     return 0.5 *(0.01 * (kullback_leibler_divergence(0.05, means)
     17                   + kullback_leibler_divergence(1 - 0.05, 1 - means)))
     18 #     return 1e-10

C:\ProgramData\Anaconda3\lib\site-packages\keras\losses.py in kullback_leibler_divergence(y_true, y_pred)
     79 
     80 def kullback_leibler_divergence(y_true, y_pred):
---> 81     y_true = K.clip(y_true, K.epsilon(), 1)
     82     y_pred = K.clip(y_pred, K.epsilon(), 1)
     83     return K.sum(y_true * K.log(y_true / y_pred), axis=-1)

C:\ProgramData\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py in clip(x, min_value, max_value)
   1599     if max_value is None:
   1600         max_value = np.inf
-> 1601     min_value = _to_tensor(min_value, x.dtype.base_dtype)
   1602     max_value = _to_tensor(max_value, x.dtype.base_dtype)
   1603     return tf.clip_by_value(x, min_value, max_value)

AttributeError: 'float' object has no attribute 'dtype'

我需要 0.05 和平均值之间的 KL 散度计算 i 上的以下总和：

KL=sum(0.05*\log(0.05/mean[i]))

Answer 1

您应该将 KL 散度正则化器更改为：

def kl_divergence_regularizer(inputs):
        means = K.mean((inputs))
        down = 0.05 * K.ones_like(means)
        up = (1 - 0.05) * K.ones_like(means)
        return 0.5 *(0.01 * (kullback_leibler_divergence(down, means)
                      + kullback_leibler_divergence(up, 1 - means)))

这是因为 KL 散度需要两个向量/张量，标量和向量之间的 KL 无法计算（未定义）。

Answer 2

kullback_leibler_divergence(0.05, means)

这是一个loss 函数。它期待y_true, y_pred，两者都是张量。 您正在传递一个float (0.05)，系统正在尝试获取这个假定张量的属性，但它不是张量。

同样的问题将发生在第二个kullback_leibler_divergence 调用中，你再次传递一个浮点数1 - 0.05 而不是张量。

一个简单的解决方案（但您必须检查这在数学上是否合理——我不知道 KL_diff 应该做什么）是使用 K.ones_like(means) * 0,05 并在第二个调用中使用 K.ones_like(means)*(1-0.05)。