Keras 是否具有将输入词向量复制并反向传播到仅一组的功能？

Question

我的目标是在 Keras 中创建一个 RNN-CNN 网络，该网络根据文本段落预测分类输出。在我当前的模型中，段落首先嵌入到特征向量中，然后输入到 2 个 cuDNNGRU 层、4 个 Conv1D 和 MaxPooling 层，然后再输入到 Dense 输出层。

但是，我找到了一个参考，其中提到了一种处理词向量的多通道方法，该方法涉及复制初始向量，通过 CNN 层运行一组，然后在池化之前将输出与副本相加。这样做是为了防止反向传播到一组向量中，从而保留原始词向量的一些语义思想。

我已尝试搜索此内容，但与多通道和 CNN 相关的唯一内容是使用多种大小的 n-gram 内核。 Keras 是否提供任何可用于实现此目的的功能？

Answer 1

是的，您可以使用函数 API 来实现。

这是一个小例子，您可以随意适应您的需要：

embed_input = Input(shape=(300,))
embedded_sequences = Embedding(10000, 10)(embed_input)
embed=SpatialDropout1D(0.5)(embedded_sequences)

gru=Bidirectional(CuDNNGRU(200, return_sequences = True))(embed)

conv=Conv1D(filters=4,
            padding = "valid",
            kernel_size=4,
            kernel_initializer='he_uniform',
            activation='relu')(gru)

avg_pool = GlobalAveragePooling1D()(conv)
max_pool = GlobalMaxPooling1D()(conv)
gru_pool = GlobalAveragePooling1D()(gru)


l_merge = concatenate([avg_pool, max_pool, gru_pool])



output = Dense(6, activation='sigmoid')(l_merge)
model = Model(embed_input, output)


model.summary()


output:

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_10 (InputLayer)           (None, 300)          0                                            
__________________________________________________________________________________________________
embedding_9 (Embedding)         (None, 300, 10)      100000      input_10[0][0]                   
__________________________________________________________________________________________________
spatial_dropout1d_9 (SpatialDro (None, 300, 10)      0           embedding_9[0][0]                
__________________________________________________________________________________________________
bidirectional_8 (Bidirectional) (None, 300, 400)     254400      spatial_dropout1d_9[0][0]        
__________________________________________________________________________________________________
conv1d_6 (Conv1D)               (None, 297, 4)       6404        bidirectional_8[0][0]            
__________________________________________________________________________________________________
global_average_pooling1d_6 (Glo (None, 4)            0           conv1d_6[0][0]                   
__________________________________________________________________________________________________
global_max_pooling1d_6 (GlobalM (None, 4)            0           conv1d_6[0][0]                   
__________________________________________________________________________________________________
global_average_pooling1d_7 (Glo (None, 400)          0           bidirectional_8[0][0]            
__________________________________________________________________________________________________
concatenate_5 (Concatenate)     (None, 408)          0           global_average_pooling1d_6[0][0] 
                                                                 global_max_pooling1d_6[0][0]     
                                                                 global_average_pooling1d_7[0][0] 
__________________________________________________________________________________________________
dense_5 (Dense)                 (None, 6)            2454        concatenate_5[0][0]              
==================================================================================================
Total params: 363,258
Trainable params: 363,258
Non-trainable params: 0

以及图的结构：