在全连接中访问变量和偏置张量答案

【问题标题】：accessing the variable and bias tensor in fully_connected在全连接中访问变量和偏置张量
【发布时间】：2018-05-31 01:31:39
【问题描述】：

我想在训练步骤中将变量和偏差张量保存为检查点。我使用了 tf.contrib.layers 中的 fully_connected() 来实现几个全连接层。为此，我需要提取那些全连接层的变量和偏差张量。怎么办？

【问题讨论】：

标签： machine-learning tensorflow neural-network artificial-intelligence

【解决方案1】：

顺便提一下：

无需提取权重和偏差来保存它们。对于 tf.layers 或 tf.contrib.layers，如果 trainable 设置为True，则权重和偏差将添加到GraphKeys.TRAINABLE_VARIABLES，这是GraphKeys.GLOBAL_VARIABLES 的子集。因此，如果您在某些时候使用saver = tf.train.Saver(var_list=tf.global_variables()) 和saver.save(sess, save_path, global_step)，权重和偏差将被保存。
如果您确实必须提取变量，一种方法是使用 tf.get_variable 或 tf.get_default_graph().get_tensor_by_name 和正确的变量名称，如另一个答案所述。
您可能已经注意到 TensorFlow 类，例如 tf.layer.Dense 和 tf.layers.Conv2D。构建完成后，它们有 weights / variables 方法返回权重和偏差张量。

【讨论】：

【解决方案2】：

tf.trainable_variables() 将为您提供网络中所有可训练变量的列表。这可以通过使用 variable_scope 和 name_scope 来改善，如下所述：How to get weights from tensorflow fully_connected

In [1]: import tensorflow as tf

In [2]: a1 = tf.get_variable(name='a1', shape=(1,2), dtype=tf.float32)

In [3]: fc = tf.contrib.layers.fully_connected(a1, 4)

In [4]: sess = tf.Session()
2017-12-17 21:09:18.127498: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-12-17 21:09:18.127554: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-12-17 21:09:18.127578: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-12-17 21:09:18.127598: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-12-17 21:09:18.127618: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.

In [5]: tf.trainable_variables()
Out[5]: 
[<tf.Variable 'a1:0' shape=(1, 2) dtype=float32_ref>,
 <tf.Variable 'fully_connected/weights:0' shape=(2, 4) dtype=float32_ref>,
 <tf.Variable 'fully_connected/biases:0' shape=(4,) dtype=float32_ref>]

In [6]: for var in tf.trainable_variables():
   ...:     if 'weights' in var.name or 'biases' in var.name:
   ...:         print(var)
   ...:         
<tf.Variable 'fully_connected/weights:0' shape=(2, 4) dtype=float32_ref>
<tf.Variable 'fully_connected/biases:0' shape=(4,) dtype=float32_ref>

In [7]:

【讨论】：