Siamese 神经网络如何计算具有三元组损失的输出之间的距离？

Question

我正在使用连体神经网络来学习文本之间的相似性。

这是我为此任务创建的 SNN 网络：它将两个输入馈送到双向 LSTM 中，该 LSTM 共享/更新权重，然后产生两个输出。然后计算这两个输出之间的距离。

    input_1 = Input(shape=(max_len,))
    input_2 = Input(shape=(max_len,))

    lstm_layer = Bidirectional(LSTM(50, dropout=0.2, recurrent_dropout=0.2)) # Won't work on GPU
    embeddings_initializer = Constant(embed_matrix)
    emb =  Embedding(len(tokenizer.word_index)+1,
                     embedding_dim,
                     embeddings_initializer=embeddings_initializer,
                     input_length=max_len,
                     weights=[embed_matrix],
                     trainable=True)

    e1 = emb(input_1)
    x1 = lstm_layer(e1)

    e2 = emb(input_2)
    x2 = lstm_layer(e2)

    mhd = lambda x: exponent_neg_cosine_distance(x[0], x[1]) 
    merged = Lambda(function=mhd, output_shape=lambda x: x[0], name='cosine_distance')([x1, x2])
    preds = Dense(1, activation='sigmoid')(merged)
    model = Model(inputs=[input_1, input_2], outputs=preds)

    model.compile(loss = "binary_crossentropy",  metrics=['acc'], optimizer = optimizer)

然而，我最近读到使用三元组损失可以改善我的 SNN。这是一个使用三元组损失进行相似性学习的 SNN 示例：

embedding_model = tf.keras.models.Sequential([
    tf.keras.Bidirectional(LSTM(50, dropout=0.2, recurrent_dropout=0.2))
    tf.keras.layers.Dense(64, activation='relu', input_shape=(784,)),
    tf.keras.layers.Dense(emb_size, activation='sigmoid')
])

input_anchor = tf.keras.layers.Input(shape=(784,))
input_positive = tf.keras.layers.Input(shape=(784,))
input_negative = tf.keras.layers.Input(shape=(784,))

embedding_anchor = embedding_model(input_anchor)
embedding_positive = embedding_model(input_positive)
embedding_negative = embedding_model(input_negative)

output = tf.keras.layers.concatenate([embedding_anchor, embedding_positive, embedding_negative], axis=1)

net = tf.keras.models.Model([input_anchor, input_positive, input_negative], output)
net.summary()

net.compile(loss=triplet_loss, optimizer=adam_optim)

def triplet_loss(y_true, y_pred, alpha = 0.4):
    """
    Implementation of the triplet loss function
    Arguments:
    y_true -- true labels, required when you define a loss in Keras, you don't need it in this function.
    y_pred -- python list containing three objects:
            anchor -- the encodings for the anchor data
            positive -- the encodings for the positive data (similar to anchor)
            negative -- the encodings for the negative data (different from anchor)
    Returns:
    loss -- real number, value of the loss
    """
    print('y_pred.shape = ',y_pred)
    
    total_lenght = y_pred.shape.as_list()[-1]
#     print('total_lenght=',  total_lenght)
#     total_lenght =12
    
    anchor = y_pred[:,0:int(total_lenght*1/3)]
    positive = y_pred[:,int(total_lenght*1/3):int(total_lenght*2/3)]
    negative = y_pred[:,int(total_lenght*2/3):int(total_lenght*3/3)]

    # distance between the anchor and the positive
    pos_dist = K.sum(K.square(anchor-positive),axis=1)

    # distance between the anchor and the negative
    neg_dist = K.sum(K.square(anchor-negative),axis=1)

    # compute loss
    basic_loss = pos_dist-neg_dist+alpha
    loss = K.maximum(basic_loss,0.0)
 
    return loss

我的困惑在于具有三元组损失的 SNN 网络。如何计算三个输出之间的距离？

在我包含的第一个 SNN 代码块中，这一行 merged = Lambda(function=mhd, output_shape=lambda x: x[0], name='cosine_distance')([x1, x2]) 正在计算两个向量之间的距离。

但是在第二个 SNN 中，我看不到在哪里/是否计算了 3 个向量之间的距离。如果不需要计算距离，为什么会这样？

Answer 1

我不太清楚你为什么在输出中连接三个嵌入向量。我建议您仔细阅读https://keras.io/examples/vision/siamese_network/ 的文档。

在那里，您会找到以下代码 sn-p：

class DistanceLayer(layers.Layer):
    """
    This layer is responsible for computing the distance between the anchor
    embedding and the positive embedding, and the anchor embedding and the
    negative embedding.
    """

    def __init__(self, **kwargs):
        super().__init__(**kwargs)

    def call(self, anchor, positive, negative):
        ap_distance = tf.reduce_sum(tf.square(anchor - positive), -1)
        an_distance = tf.reduce_sum(tf.square(anchor - negative), -1)
        return (ap_distance, an_distance)


anchor_input = layers.Input(name="anchor", shape=target_shape + (3,))
positive_input = layers.Input(name="positive", shape=target_shape + (3,))
negative_input = layers.Input(name="negative", shape=target_shape + (3,))

distances = DistanceLayer()(
    embedding(resnet.preprocess_input(anchor_input)),
    embedding(resnet.preprocess_input(positive_input)),
    embedding(resnet.preprocess_input(negative_input)),
)

siamese_network = Model(
    inputs=[anchor_input, positive_input, negative_input], outputs=distances
)

如您所见，它们将嵌入发送到 DistanceLayer 类，在该类中计算正负距离，然后作为元组返回，该元组将放置在模型的输出中。