Keras：如何连接两个 CNN？

Question

我正在尝试实现本文中的CNN模型（https://arxiv.org/abs/1605.07333）

在这里，它们有两个不同的上下文作为输入，由两个独立的卷积层和最大池化层处理。合并后，他们将结果连接起来。

假设每个CNN都是这样建模的，我该如何实现上面的模型？

def baseline_cnn(activation='relu'):

model = Sequential()
model.add(Embedding(SAMPLE_SIZE, EMBEDDING_DIMS, input_length=MAX_SMI_LEN))
model.add(Dropout(0.2))
model.add(Conv1D(NUM_FILTERS, FILTER_LENGTH, padding='valid', activation=activation, strides=1))
model.add(GlobalMaxPooling1D())
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam',  metrics=['accuracy'])

return model

提前致谢！

最终代码：我只是使用了@FernandoOrtega 的解决方案：

def build_combined(FLAGS, NUM_FILTERS, FILTER_LENGTH1, FILTER_LENGTH2):
    Dinput = Input(shape=(FLAGS.max_dlen, FLAGS.dset_size))
    Tinput = Input(shape=(FLAGS.max_tlen, FLAGS.tset_size))


    encode_d= Conv1D(filters=NUM_FILTERS, kernel_size=FILTER_LENGTH1,  activation='relu', padding='valid',  strides=1)(Dinput)
    encode_d = Conv1D(filters=NUM_FILTERS*2, kernel_size=FILTER_LENGTH1,  activation='relu', padding='valid',  strides=1)(encode_d)
    encode_d = GlobalMaxPooling1D()(encode_d)

    encode_tt = Conv1D(filters=NUM_FILTERS, kernel_size=FILTER_LENGTH2,  activation='relu', padding='valid',  strides=1)(Tinput)
    encode_tt = Conv1D(filters=NUM_FILTERS*2, kernel_size=FILTER_LENGTH1,  activation='relu', padding='valid',  strides=1)(encode_tt)
    encode_tt = GlobalMaxPooling1D()(encode_tt)

    encode_combined = keras.layers.concatenate([encode_d, encode_tt])


    # Fully connected 
    FC1 = Dense(1024, activation='relu')(encode_combined)
    FC2 = Dropout(0.1)(FC1)
    FC2 = Dense(512, activation='relu')(FC2)

    predictions = Dense(1, kernel_initializer='normal')(FC2) 

    combinedModel = Model(inputs=[Dinput, Tinput], outputs=[predictions])
    combinedModel.compile(optimizer='adam', loss='mean_squared_error', metrics=[accuracy])

    print(combinedModel.summary())

    return combinedModel

Answer 1

如果你想连接两个子网络，你应该使用 keras.layer.concatenate 函数。

此外，我建议您应该使用功能 API，只要它最容易设计出像您这样的复杂网络。例如：

def baseline_cnn(activation='relu')

    # Defining input 1
    input1 = Embedding(SAMPLE_SIZE, EMBEDDING_DIMS, input_length=MAX_SMI_LEN)
    x1 = Dropout(0.2)(input)
    x1 = Conv1D(NUM_FILTERS, FILTER_LENGTH, padding='valid', activation=activation, strides=1)(x1)
    x1 = GlobalMaxPooling1D()(x1)

    # Defining input 2
    input2 = Embedding(SAMPLE_SIZE, EMBEDDING_DIMS, input_length=MAX_SMI_LEN)
    x2 = Dropout(0.2)(input)
    x2 = Conv1D(NUM_FILTERS, FILTER_LENGTH, padding='valid', activation=activation, strides=1)(x2)
    x2 = GlobalMaxPooling1D()(x2)

    # Merging subnetworks
    x = concatenate([input1, input2])

    # Final Dense layer and compilation
    x = Dense(1, activation='sigmoid')
    model = Model(inputs=[input1, input2], x)
    model.compile(loss='binary_crossentropy', optimizer='adam',  metrics=['accuracy'])

return model

编译此模型后，您可以通过model.fit([data_split1, data_split2]) 拟合/评估它，其中data_split1 和data_split2 是您作为输入的不同上下文。

Keras 文档中有关多输入的更多信息：Multi-input and multi-output models。