LSTM 层不接受 CNN 层输出的输入形状

Question

我正在尝试创建一个 CNN + LSTM 网络，但 LSTM 层不接受输入形状。有什么我可以做的吗？

model = Sequential()
model.add(Conv2D(128, (2,2), padding = 'same', input_shape=(30, 216, 1)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.2))

model.add(Conv2D(256, (2,2), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.2))

model.add(LSTM(512, input_shape = (7, 54, 256,)))
model.add(Flatten())
model.add(Dense(7, activation='softmax'))

ValueError：lstm_21 层的输入 0 与该层不兼容：预期 ndim=3，发现 ndim=4。收到的完整形状：[None, 7, 54, 256]

Answer 1

Keras 中的LSTM 层需要这种格式作为输入：

inputs: A 3D tensor with shape [batch, timesteps, feature].

因此，您不能直接传递非循环层。首先，Flatten() 之前的层，然后将该层包裹到 TimeDistributed 层中，如下所示：

model.add(TimeDistributed(Flatten()))
model.add(LSTM(8))

这个TimeDistributed 层允许对输入的每个时间切片应用层。这是一个完整的示例：

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, LSTM, \
    Dense, Flatten, Dropout, MaxPooling2D, Activation, TimeDistributed
import numpy as np

X = np.random.rand(100, 30, 216, 1)
y = np.random.randint(0, 7, 100)

model = Sequential()
model.add(Conv2D(16, (2,2), padding = 'same', input_shape=(30, 216, 1)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.2))

model.add(Conv2D(32, (2,2), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.2))
model.add(TimeDistributed(Flatten()))
model.add(LSTM(8))
model.add(Dense(7, activation='softmax'))

model.compile(loss='sparse_categorical_crossentropy', optimizer='adam')
history = model.fit(X, y)