将 2 个模型的输出相乘，并将该层用作第三个模型的输出

Question

我连接了 2 个模型：1 个是动态模型（基于参数的变化）和 3 层的静态模型，我必须将它们关联起来。我的第一个测试是连接模型：

def __make_model(
    channel_counts=[],
    kernel_sizes=[],
    dilation_rates=[],
    pool_sizes=[],
    dropout_rates=[],
    residual_blocks=0,
    activation_function=None,
    data_shape=None,
):
    """
    Assembles the Keras Model.
    Args:
        channel_counts: Tuple of the number of channels in each layer; the length of
            the tuple defines the number of convolutional layers
        kernel_sizes: Respective kernel sizes of each convolutional layer;
            padded with 3s, if less than channel_counts are given
        dilation_rates: Respective dilation rates of each convolutional layer;
            padded with 1s, if less than channel_counts are given
        pool_sizes: Respective sizes for max pooling after each convolutional layer;
            padded with 1s, if less than channel_counts are given
        dropout_rates: Respective rate of dropout to apply before each conv layer
        residual_blocks: Times to repeat convolutions as residual block (no pooling)
        activation_function: Name of the activation function applied at each neuron
        data_shape: Tuple defining the (non-batch) shape of the input to the model
    Returns: The uncompiled Keras Model
    """
    # Assemble the model.
    input_tensor = Input(batch_shape=data_shape)
    tensor = input_tensor

    tensor = block_of_convolutions(
        tensor,
        channel_counts=channel_counts,
        kernel_sizes=kernel_sizes,
        dilation_rates=dilation_rates,
        pool_sizes=pool_sizes,
        dropout_rates=dropout_rates,
        activation_function=activation_function,
    )

    if channel_counts[-1] != 1:
        layer = Conv2D(
            filters=1,
            kernel_size=3,
            padding="same",
            activation=activation_function,
        )
        tensor = layer(tensor)

    argmax_layer = CenterOfMass(normalize_output=True, name="center_of_mass")
    tensor = argmax_layer(tensor)

    model_COM = Model(
        inputs=input_tensor, outputs=tensor, name="model_Center_of_mass"
    )

    input_corners = Input(batch_shape=(data_shape[0], 3), name="input_corners_data")
    model_corners = Model(
        inputs=input_corners, outputs=input_corners, name="model_corners_data"
    )

    concat = Concatenate(axis=1)

    combined = concat([model_COM.output, model_corners.output])

    z = Dense(16, name="dense_final")(combined)
    z = Dense(2, name="direction")(z)

    model = Model(
        inputs=[model_COM.input, model_corners.input],
        outputs=z,
        name="top_view_director",
    )

    return model

这个模型的结果运行良好，但是当我用乘法层修改 Contatenation 层时。

    model_COM = Model(
        inputs=input_tensor, outputs=tensor, name="model_Center_of_mass"
    )

    input_corners = Input(batch_shape=(data_shape[0], 3), name="input_corners_data")
    layer_corners= Dense(16, name="dense_final", activation = "relu")(input_corners)
    output_corners = Dense(2, name="direction", activation = "relu")(layer_corners)       
    
    model_corners = Model(
        inputs=input_corners, outputs=output_corners, name="model_corners_data"
    )

    combined = Multiply(name="multiply")([model_COM.output, model_corners.output])
    output = Dense(2, name="final_output")(combined) 

    model = Model(
        inputs=[model_COM.input, model_corners.input],
        outputs=output,
        name="top_view_director",
    )

    return model

我收到以下异常：

ValueError: Found unexpected keys that do not correspond to any Model output: dict_keys(['direction']). Expected: ['final_output']

我也尝试将乘法层作为最后一层，但我收到了类似的错误

Found unexpected keys that do not correspond to any Model output: dict_keys(['direction']). Expected: ['multiply']

...当我删除模型并仅使用模型的输入时（就好像它是单个模型一样）

...
input_tensor = Input(batch_shape=data_shape)
...
tensor = argmax_layer(tensor) # Output that contains the model itself.
input_corners = Input(batch_shape=(data_shape[0], 3), name="input_corners_data")
...
combined = Multiply(name="multiply")([tensor, output_corners])
    #output = Dense(2, name="final_output")(combined) 

    model = Model(
        inputs=[input_tensor, input_corners],
        outputs=combined,
        name="top_view_director",
    )

    return model

然后是类似的异常

ValueError: Found unexpected keys that do not correspond to any Model output: dict_keys(['direction']). Expected: ['multiply']

有什么我忘了在模型中写的吗？

我不能（应该/不得）使用 keras.dot，因为 model_corners 的输入与另一个输出一对一地相乘 [(x_corners * x_com),(y_corners * y_com)]

Answer 1

抱歉，我只需要导入 keras.layers.multiply 而不是 keras.layers.Multiply（我没想到会这样）

...
input_tensor = Input(batch_shape=data_shape)
...
tensor = argmax_layer(tensor) # Output that contains the model itself.
input_corners = Input(batch_shape=(data_shape[0], 3), name="input_corners_data")
...
combined = multiply([tensor, output_corners])

    model = Model(
        inputs=[input_tensor, input_corners],
        outputs=combined,
        name="top_view_director",
    )

    return model