比较 Conv2D 与 Tensorflow 和 PyTorch 之间的填充

Question

我正在尝试将从 Tensorflow 模型中保存的权重导入 PyTorch。到目前为止，结果非常相似。当模型使用stride=2 调用conv2d 时，我遇到了障碍。

为了验证不匹配，我在 TF 和 PyTorch 之间进行了非常简单的比较。首先，我将conv2d 与stride=1 进行比较。

import tensorflow as tf
import numpy as np
import torch
import torch.nn.functional as F


np.random.seed(0)
sess = tf.Session()

# Create random weights and input
weights = torch.empty(3, 3, 3, 8)
torch.nn.init.constant_(weights, 5e-2)
x = np.random.randn(1, 3, 10, 10)

weights_tf = tf.convert_to_tensor(weights.numpy(), dtype=tf.float32)
# PyTorch adopts [outputC, inputC, kH, kW]
weights_torch = torch.Tensor(weights.permute((3, 2, 0, 1)))

# Tensorflow defaults to NHWC
x_tf = tf.convert_to_tensor(x.transpose((0, 2, 3, 1)), dtype=tf.float32)
x_torch = torch.Tensor(x)

# TF Conv2D
tf_conv2d = tf.nn.conv2d(x_tf,
                         weights_tf,
                         strides=[1, 1, 1, 1],
                         padding="SAME")

# PyTorch Conv2D
torch_conv2d = F.conv2d(x_torch, weights_torch, padding=1, stride=1)

sess.run(tf.global_variables_initializer())
tf_result = sess.run(tf_conv2d)

diff = np.mean(np.abs(tf_result.transpose((0, 3, 1, 2)) - torch_conv2d.detach().numpy()))
print('Mean of Abs Diff: {0}'.format(diff))

本次执行的结果是：

Mean of Abs Diff: 2.0443112092038973e-08

当我将stride 更改为 2 时，结果开始变化。

# TF Conv2D
tf_conv2d = tf.nn.conv2d(x_tf,
                         weights_tf,
                         strides=[1, 2, 2, 1],
                         padding="SAME")

# PyTorch Conv2D
torch_conv2d = F.conv2d(x_torch, weights_torch, padding=1, stride=2)

本次执行的结果是：

Mean of Abs Diff: 0.2104552686214447

根据 PyTorch 文档，conv2d uses zero-padding 由 padding 参数定义。因此，在我的示例中，输入的左侧、顶部、右侧和底部都添加了零。

如果 PyTorch 只是根据输入参数在两侧添加填充，在 Tensorflow 中应该很容易复制。

# Manually add padding - consistent with PyTorch
paddings = tf.constant([[0, 0], [1, 1], [1, 1], [0, 0]])
x_tf = tf.convert_to_tensor(x.transpose((0, 2, 3, 1)), dtype=tf.float32)
x_tf = tf.pad(x_tf, paddings, "CONSTANT")

# TF Conv2D
tf_conv2d = tf.nn.conv2d(x_tf,
                         weights_tf,
                         strides=[1, 2, 2, 1],
                         padding="VALID")

这个比较的结果是：

Mean of Abs Diff: 1.6035047067930464e-08

这告诉我的是，如果我能够以某种方式将默认填充行为从 Tensorflow 复制到 PyTorch 中，那么我的结果将是相似的。

This question 检查了 TensorFlow 中的填充行为。 TF documentation explains how padding is added for "SAME" convolutions.我在写这个问题时发现了这些链接。

现在我知道了 TensorFlow 的填充策略，我可以在 PyTorch 中实现它了。

Answer 1

为了复制该行为，填充大小按照 Tensorflow 文档中的说明进行计算。在这里，我通过设置 stride=2 并填充 PyTorch 输入来测试填充行为。

import tensorflow as tf
import numpy as np
import torch
import torch.nn.functional as F


np.random.seed(0)
sess = tf.Session()

# Create random weights and input
weights = torch.empty(3, 3, 3, 8)
torch.nn.init.constant_(weights, 5e-2)
x = np.random.randn(1, 3, 10, 10)

weights_tf = tf.convert_to_tensor(weights.numpy(), dtype=tf.float32)
weights_torch = torch.Tensor(weights.permute((3, 2, 0, 1)))

# Tensorflow padding behavior. Assuming that kH == kW to keep this simple.
stride = 2
if x.shape[2] % stride == 0:
    pad = max(weights.shape[0] - stride, 0)
else:
    pad = max(weights.shape[0] - (x.shape[2] % stride), 0)

if pad % 2 == 0:
    pad_val = pad // 2
    padding = (pad_val, pad_val, pad_val, pad_val)
else:
    pad_val_start = pad // 2
    pad_val_end = pad - pad_val_start
    padding = (pad_val_start, pad_val_end, pad_val_start, pad_val_end)

x_tf = tf.convert_to_tensor(x.transpose((0, 2, 3, 1)), dtype=tf.float32)
x_torch = torch.Tensor(x)
x_torch = F.pad(x_torch, padding, "constant", 0)

# TF Conv2D
tf_conv2d = tf.nn.conv2d(x_tf,
                         weights_tf,
                         strides=[1, stride, stride, 1],
                         padding="SAME")

# PyTorch Conv2D
torch_conv2d = F.conv2d(x_torch, weights_torch, padding=0, stride=stride)

sess.run(tf.global_variables_initializer())
tf_result = sess.run(tf_conv2d)

diff = np.mean(np.abs(tf_result.transpose((0, 3, 1, 2)) - torch_conv2d.detach().numpy()))
print('Mean of Abs Diff: {0}'.format(diff))

输出是：

Mean of Abs Diff: 2.2477470551507395e-08

当我开始写这个问题时，我不太清楚为什么会发生这种情况，但是阅读一下很快就澄清了这一点。我希望这个例子可以帮助其他人。