有没有办法将keras mobilenetv2模型划分为子模型？

Question

我正在尝试将mobilenetv2 模型分成两部分。

出于某些原因，我首先想运行模型的第一部分，保存输出，然后将其提供给第二个模型。我试过找到here的代码， 但我收到以下错误：

ValueError: A merge layer should be called on a list of inputs.

我认为这是因为模型不是 Sequential。 有人可以帮忙吗？

Answer 1

正如我在 cmets 中提到的，mobile_net_v2 中的某些层需要多个输入，这些输入是其他一些先前层的输出。因此，将它们单独添加到 sequential 模型会导致错误。我有一个替代解决方案给你。使用 this 链接中的 mobile_net_v2 实现（我自己的），我能够创建您想要的模型：

import tensorflow as tf
from tensorflow.keras import layers, Model, Sequential


def conv_block(input_tensor, c, s, t, expand=True):
    """
    Convolutional Block for mobile net v2
    Args:
        input_tensor (keras tensor): input tensor
        c (int): output channels
        s (int): stride size of first layer in the series
        t (int): expansion factor
        expand (bool): expand filters or not?

    Returns: keras tensor
    """
    first_conv_channels = input_tensor.get_shape()[-1]
    if expand:
        x = layers.Conv2D(
            first_conv_channels*t,
            1,
            1,
            padding='same',
            use_bias=False
        )(input_tensor)
        x = layers.BatchNormalization()(x)
        x = layers.ReLU(6.0)(x)
    else:
        x = input_tensor

    x = layers.DepthwiseConv2D(
        3,
        s,
        'same',
        1,
        use_bias=False
    )(x)
    x = layers.BatchNormalization()(x)
    x = layers.ReLU(6.0)(x)

    x = layers.Conv2D(
        c,
        1,
        1,
        padding='same',
        use_bias=False
    )(x)
    x = layers.BatchNormalization()(x)

    if input_tensor.get_shape() == x.get_shape() and s == 1:
        return x+input_tensor

    return x


def splitted_model(input_shape=(224,224,3)):

    input = layers.Input(shape=input_shape)

    x = layers.Conv2D(
        32,
        3,
        2,
        padding='same',
        use_bias=False
    )(input)
    x = layers.BatchNormalization()(x)
    x = layers.ReLU(6.0)(x)

    x = conv_block(x, 16, 1, 1, expand=False)
    x = conv_block(x, 24, 2, 6)
    x = conv_block(x, 24, 1, 6)

    x = conv_block(x, 32, 2, 6)
    x = conv_block(x, 32, 1, 6)
    x = conv_block(x, 32, 1, 6)

    x = conv_block(x, 64, 2, 6)
    x = conv_block(x, 64, 1, 6)
    x = conv_block(x, 64, 1, 6)
    x = conv_block(x, 64, 1, 6)

    model_f = Model(inputs=input, outputs=x)

    input_2 = layers.Input(shape=(x.shape[1:]))
    x = conv_block(input_2, 96, 1, 6)
    x = conv_block(x, 96, 1, 6)
    x = conv_block(x, 96, 1, 6)

    x = conv_block(x, 160, 2, 6)
    x = conv_block(x, 160, 1, 6)
    x = conv_block(x, 160, 1, 6)

    x = conv_block(x, 320, 1, 6)

    x = layers.Conv2D(
        1280,
        1,
        1,
        padding='same',
        use_bias=False
    )(x)
    x = layers.BatchNormalization()(x)
    x = layers.ReLU(6.0)(x)

    x = layers.GlobalAveragePooling2D()(x)


    model_h = Model(inputs=input_2, outputs=x)

    return model_f, model_h

您可以这样创建两个模型：

IMG_SIZE = 160
IMG_SHAPE = (IMG_SIZE, IMG_SIZE, 3)
model_f, model_h = splitted_model(input_shape=IMG_SHAPE)

请注意，权重是随机初始化的。如果您想在 imagenet 上训练来自 mobilenet_v2 的权重，您可以运行以下代码来复制权重：

mobile_net = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,
                                           include_top=False,
                                           weights='imagenet')
layer_f_counter = 0
layer_h_counter = 0
for i in range(len(mobile_net.layers)):
  if layer_f_counter<len(model_f.layers):
    if len(mobile_net.layers[i].get_weights()) > 0:
      if len(model_f.layers[layer_f_counter].get_weights()) > 0:
        print(mobile_net.layers[i].name,'here', model_f.layers[layer_f_counter].name, layer_f_counter)
        model_f.layers[layer_f_counter].set_weights(mobile_net.layers[i].get_weights())
      layer_f_counter += 1
      print(layer_f_counter)
    else:
      if len(model_f.layers[layer_f_counter].get_weights()) > 0:
        continue
      else:
        layer_f_counter+=1

  else:
    if layer_h_counter<len(model_h.layers):
      if len(mobile_net.layers[i].get_weights()) > 0:
        if len(model_h.layers[layer_h_counter].get_weights()) > 0:
          print(mobile_net.layers[i].name,'here', model_h.layers[layer_h_counter].name, layer_h_counter)
          model_h.layers[layer_h_counter].set_weights(mobile_net.layers[i].get_weights())
        layer_h_counter += 1
        print(layer_h_counter)
      else:
        if len(model_h.layers[layer_h_counter].get_weights()) > 0:
          continue
        else:
          layer_h_counter+=1

它遍历从 Keras 加载的 mobilenet_v2 层，将第一部分的权重复制到 model_f，其余复制到model_h。您可以通过从 mobile_net 打印出一些随机层权重以及以下新模型来检查权重是否正确复制：

print(model_f.layers[1].get_weights()) # printing weights of first conv layer in model_f
print(mobile_net.get_layer('Conv1').get_weights()) # printing weights of fist conv layer in mobile_net

同样适用于model_h：

print(model_h.layers[-4].get_weights()) # printing weights of last conv layer in model_h
print(mobile_net.get_layer('Conv_1').get_weights()) # printing weights of last conv layer in mobile_net

请注意，我随机选择了将 moile_net 分成 model_f 和 model_h 的块，您可以对其进行编辑以更改要拆分的位置。希望对您有所帮助。