将权重初始化器与 tf.nn.conv2d 一起使用

Question

使用tf.layers.conv2d时，设置初始化器很容易，可以通过它的参数来完成。但是如果我使用tf.nn.conv2d 会怎样？我使用此代码。这是否等同于在tf.layers.conv2d 中设置kernel_initializer 参数？虽然程序运行没有错误，但我不知道如何验证它是否按预期执行。

 with tf.name_scope('conv1_2') as scope:
        kernel = tf.get_variable(initializer=tf.contrib.layers.xavier_initializer(), 
                                 shape=[3, 3, 32, 32], name='weights')
        conv = tf.nn.conv2d(conv1_1, kernel, [1, 1, 1, 1], padding='SAME')
        biases = tf.Variable(tf.constant(0.0, shape=[32], dtype=tf.float32),
                             trainable=True, name='biases')
        out = tf.nn.bias_add(conv, biases)
        self.conv1_2 = tf.nn.relu(out, name=scope)
        self.parameters += [kernel, biases]

Answer 1

下面的操作是一样的（见here）。

至于内核及其初始化，我瞥了一眼代码，它看起来相同...layers.conv2d 在一天结束时调用了tf.get_variable。

但我想凭经验看看，所以这里有一个测试代码，它使用每种方法（tf.layers.conv2d 和tf.nn.conv2d）声明一个 conv2d，评估初始化的内核并比较它们。

我已经任意设置了不应干扰比较的内容，例如输入张量和步幅。

import tensorflow as tf
import numpy as np


# the way you described in your question
def _nn(input_tensor, initializer, filters, size):
    kernel = tf.get_variable(
        initializer=initializer, 
        shape=[size, size, 32, filters],
        name='kernel')

    conv = tf.nn.conv2d(
        input=input_tensor,
        filter=kernel,
        strides=[1, 1, 1, 1],
        padding='SAME')

    return kernel

# the other way
def _layer(input_tensor, initializer, filters, size):
    tf.layers.conv2d(
        inputs=input_tensor,
        filters=filters,
        kernel_size=size,
        kernel_initializer=initializer)

    # 'conv2d/kernel:0' is the name of the generated kernel
    return tf.get_default_graph().get_tensor_by_name('conv2d/kernel:0')

def _get_kernel(method):
    # an isolated context for each conv2d
    graph = tf.Graph()
    sess = tf.Session(graph=graph)

    with graph.as_default(), sess.as_default():
        # important so that same randomness doesnt play a role
        tf.set_random_seed(42)

        # arbitrary input tensor with compatible shape
        input_tensor = tf.constant(1.0, shape=[1, 64, 64, 32])

        initializer = tf.contrib.layers.xavier_initializer()

        kernel = method(
            input_tensor=input_tensor,
            initializer=initializer,
            filters=32,
            size=3)

        sess.run(tf.global_variables_initializer())
        return sess.run(kernel)

if __name__ == '__main__':
    kernel_nn = _get_kernel(_nn)
    kernel_layer = _get_kernel(_layer)

    print('kernels are ', end='')
    # compares shape and values
    if np.array_equal(kernel_layer, kernel_nn):
        print('exactly the same')
    else:
        print('not the same!')

输出是... 内核完全相同。

文档，顺便说一句：tf.nn.conv2d 和 tf.layers.conv2d。