如何实现“多向”LSTM？

Question

我正在尝试从论文“Dropout 改进用于手写识别的递归神经网络”中实现这个 LSTM 架构：

在论文中，研究人员将多向 LSTM 层定义为“并行应用的四个 LSTM 层，每个层都有一个特定的扫描方向”

以下是（我认为）网络在 Keras 中的样子：

from keras.layers import LSTM, Dropout, Input, Convolution2D, Merge, Dense, Activation, TimeDistributed
from keras.models import Sequential

def build_lstm_dropout(inputdim, outputdim, return_sequences=True, activation='tanh'):
    net_input = Input(shape=(None, inputdim))
    model = Sequential()
    lstm  = LSTM(output_dim=outputdim, return_sequences=return_sequences, activation=activation)(net_input)
    model.add(lstm)
    model.add(Dropout(0.5))
    return model

def build_conv(nb_filter, nb_row, nb_col, net_input, border_mode='relu'):
    return TimeDistributed(Convolution2D( nb_filter, nb_row, nb_col, border_mode=border_mode, activation='relu')(net_input))

def build_lstm_conv(lstm, conv):
    model = Sequential()
    model.add(lstm)
    model.add(conv)
    return model

def build_merged_lstm_conv_layer(lstm_conv, mode='concat'):
    return Merge([lstm_conv, lstm_conv, lstm_conv, lstm_conv], mode=mode)

def build_model(feature_dim, loss='ctc_cost_for_train', optimizer='Adadelta'):
    net_input = Input(shape=(1, feature_dim, None))

    lstm = build_lstm_dropout(2, 6)
    conv = build_conv(64, 2, 4, net_input)

    lstm_conv = build_lstm_conv(lstm, conv)

    first_layer = build_merged_lstm_conv_layer(lstm_conv)

    lstm = build_lstm_dropout(10, 20)
    conv = build_conv(128, 2, 4, net_input)

    lstm_conv = build_lstm_conv(lstm, conv)

    second_layer = build_merged_lstm_conv_layer(lstm_conv)

    lstm = build_lstm_dropout(50, 1)
    fully_connected = Dense(1, activation='sigmoid')

    lstm_fc = Sequential()
    lstm_fc.add(lstm)
    lstm_fc.add(fully_connected)

    third_layer = Merge([lstm_fc, lstm_fc, lstm_fc, lstm_fc], mode='concat')

    final_model = Sequential()
    final_model.add(first_layer)
    final_model.add(Activation('tanh'))
    final_model.add(second_layer)
    final_model.add(Activation('tanh'))
    final_model.add(third_layer)

    final_model.compile(loss=loss, optimizer=optimizer, sample_weight_mode='temporal')

    return final_model

这是我的问题：

1. 如果我的架构实现是正确的，你如何 实现四个 LSTM 层的扫描方向？
2. 如果我的实现不正确，是否可以实现 Keras 有这样的架构吗？如果没有，是否有任何其他框架可以帮助我实现这样的架构？

Answer 1

您可以查看this 了解双向 LSTM 的实现。基本上，您只需为后向 LSTM 设置 go_backwards=True。

但是，在您的情况下，您必须编写一个“镜像”+重塑层来反转行。镜像层可能看起来像（为方便起见，我在这里使用 lambda 层）：Lambda(lambda x: x[:,::-1,:])