添加的层必须是类层的实例

Question

我正在构建一个 Bi-LSTM 网络，其中包含一个注意力层。但是添加的层必须是类层的实例会给出错误。

我已经导入的一些库是

from keras.models import Model, Sequential
from keras.layers import LSTM, Activation, Dense, Dropout, Input, Embedding, Bidirectional, Conv1D, Flatten, GlobalMaxPooling1D, SpatialDropout1D
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras import backend as K
from tensorflow.keras.layers import *

注意层类是

class attention(Layer):
    
    def __init__(self, return_sequences=True):
        self.return_sequences = return_sequences
        super(attention,self).__init__()
        
    def build(self, input_shape):
        
        self.W=self.add_weight(name="att_weight", shape=(input_shape[-1],1),
                               initializer="normal")
        self.b=self.add_weight(name="att_bias", shape=(input_shape[1],1),
                               initializer="zeros")
        
        super(attention,self).build(input_shape)
        
    def call(self, x):
        
        e = K.tanh(K.dot(x,self.W)+self.b)
        a = K.softmax(e, axis=1)
        output = x*a
        
        if self.return_sequences:
            return output
        
        return K.sum(output, axis=1)

模型是这样的

model = Sequential()
model.add(Embedding(max_words, 1152, input_length=max_len, weights=[embeddings]))
model.add(BatchNormalization())
model.add(Activation('tanh'))
model.add(Dropout(0.5))
model.add(Bidirectional(LSTM(32, return_sequences=True)))
model.add(attention(return_sequences=True))
model.add(BatchNormalization())
model.add(Activation('tanh'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.summary()

但它给出了一个错误

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-41-ba5b52fe2c87> in <module>()
      1 model = Sequential()
----> 2 model.add(Embedding(max_words, 1152, input_length=max_len, weights=[embeddings]))
      3 model.add(BatchNormalization())
      4 model.add(Activation('tanh'))
      5 #model.add(SpatialDropout1D(0.5))

/usr/local/lib/python3.6/dist-packages/keras/engine/sequential.py in add(self, layer)
    131             raise TypeError('The added layer must be '
    132                             'an instance of class Layer. '
--> 133                             'Found: ' + str(layer))
    134         self.built = False
    135         if not self._layers:

TypeError: The added layer must be an instance of class Layer. Found: <tensorflow.python.keras.layers.embeddings.Embedding object at 0x7f0da41aec50>

Answer 1

This documentation page 声明在定义自定义Layer 时，应使用以下语法：

class Linear(tf.keras.layers.Layer):
    def __init__(self, units=32, input_dim=32):
        super(Linear, self).__init__()
        w_init = tf.random_normal_initializer()
        self.w = tf.Variable(
            initial_value=w_init(shape=(input_dim, units), dtype="float32"),
            trainable=True,
        )
        b_init = tf.zeros_initializer()
        self.b = tf.Variable(
            initial_value=b_init(shape=(units,), dtype="float32"), trainable=True
        )

    def call(self, inputs):
        return tf.matmul(inputs, self.w) + self.b

所以，您的 Layer 导入在技术上是正确的。但是，您使用纯keras 层初始化模型，这会导致错误。在任何地方使用tf.keras 功能，错误就会消失，例如：

https://www.tensorflow.org/guide/keras/sequential_model

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

# Your own layer here
class AttentionCustom(layers.Layer):
     pass

model = keras.Sequential(
    [
        layers.Dense(2, activation="relu", name="layer1"),
        layers.Dense(3, activation="relu", name="layer2"),
        Attention(),
        layers.Dense(4, name="layer3"),
    ]
)