如何通过围绕该张量的某些值选择窗口来从给定张量创建子张量？

Question

我的问题类似于here 提出的问题。不同之处在于我想要一个新的张量B，它是从初始张量A 中选择的一些窗口的串联。 目标是使用先验未知的张量，即：输入层。这是一个使用已定义常量的示例，只是为了解释我想做的事情：

给定 2 个 3-dim 嵌入的输入张量：

A = K.constant([[1, 1, 1], [2, 2, 2], [3, 3, 3], [4, 4, 4], [5, 5, 5], [6, 6, 6], [7, 7, 7], [2, 2, 2], [8, 8, 8], [9, 9, 9], [10, 10, 10]])
t = K.constant([[2, 2, 2], [6, 6, 6], [10, 10, 10]])

我想创建一个张量B，它是从A 中选择的以下子张量（或窗口）的串联，并且对应于t 中每个元素的出现邻域：

# windows of 3 elements, each window is a neighbourhood of a corresponding element in t
window_t_1 = [[1, 1, 1], [2, 2, 2], [3, 3, 3]]  # 1st neighbourhood of [2, 2, 2] 
window_t_2 = [[7, 7, 7], [2, 2, 2], [8, 8, 8]]  # 2nd neighbourhood of [2, 2, 2] (because it has 2 occurences in A)
window_t_3 = [[5, 5, 5], [6, 6, 6], [7, 7, 7]]  # unique neighbourhood of [6, 6, 6]
window_t_4 = [[8, 8, 8], [9, 9, 9], [10, 10, 10]]  # unique neighbourhood of [10, 10, 10]
# B must contain these selected widows:
B = [[1, 1, 1], [2, 2, 2], [3, 3, 3], [7, 7, 7], [2, 2, 2], [8, 8, 8], [5, 5, 5], [6, 6, 6], [7, 7, 7], [8, 8, 8], [9, 9, 9], [10, 10, 10]]

我们的目标是应用这个过程来重新制定我的模型的 Input 张量，而不是预定义的常量。那么，鉴于我的模型的两个输入，我该如何做到这一点：

in_A = Input(shape=(10,), dtype="int32")
in_t = Input(shape=(3,), dtype="int32")
embed_A = Embedding(...)(in_A)
embed_t = Embedding(...)(in_t)
B = ...  # some function or layer to create the tensor B as described in the example above using embed_A and embed_t
# B will be used then on the next layer like this:
# next_layer = some_other_layer(...)(embed_t, B)

或者选择子张量元素然后应用嵌入层：

in_A = Input(shape=(10,), dtype="int32")
in_t = Input(shape=(3,), dtype="int32")
B = ...  # some function to select the desired element windows as described above
embed_B = Embedding(...)(B)
embed_t = Embedding(...)(in_t)
# then add the next layer like this:
# next_layer = some_other_layer(...)(embed_t, embed_B)

提前致谢。

Answer 1

import tensorflow as tf
from tensorflow.contrib import autograph
# you can uncomment next line to enable eager execution to see what happens at each step, you'd better use the up-to-date tf-nightly to run this code
# tf.enable_eager_execution()
A = tf.constant([[1, 1, 1],
                 [2, 2, 2],
                 [3, 3, 3],
                 [4, 4, 4],
                 [5, 5, 5],
                 [6, 6, 6],
                 [7, 7, 7],
                 [2, 2, 2],
                 [8, 8, 8],
                 [9, 9, 9],
                 [10, 10, 10]])

t = tf.constant([[2, 2, 2],
                 [6, 6, 6],
                 [10, 10, 10]])

# expand A in axis 1 to compare elements in A and t with broadcast
expanded_a = tf.expand_dims(A, axis=1)

# find where A and t are equal with each other
equal = tf.equal(expanded_a, t)
reduce_all = tf.reduce_all(equal, axis=2)
# find the indices
where = tf.where(reduce_all)
where = tf.cast(where, dtype=tf.int32)

# here we want to a function to find the indices to do tf.gather, if a match 
# is found in the start or
# end of A, then pick up the two elements after or before it, otherwise the 
# left one and the right one along with itself are used
@autograph.convert()
def _map_fn(x):
    if x[0] == 0:
        return tf.range(x[0], x[0] + 3)
    elif x[0] == tf.shape(A)[0] - 1:
        return tf.range(x[0] - 2, x[0] + 1)
    else:
        return tf.range(x[0] - 1, x[0] + 2)


indices = tf.map_fn(_map_fn, where, dtype=tf.int32)

# reshape the found indices to a vector
reshape = tf.reshape(indices, [-1])

# gather output with found indices
output = tf.gather(A, reshape)

只要看懂这段代码，就可以轻松编写自定义层