TensorFlow 简单示例帮助 - 自定义渐变答案

【问题标题】：TensorFlow simple example help - custom gradientTensorFlow 简单示例帮助 - 自定义渐变
【发布时间】：2019-12-14 23:30:09
【问题描述】：

如何将自定义梯度传递给 TensorFlow 中的梯度优化函数。

我已经用一个简单的例子说明了我想要做什么（试图最小化 z = 2x^2 + y^2 + 2）。

我一直在看： https://www.tensorflow.org/api_docs/python/tf/train/Optimizer

如果你传入optimizer = tf.train.GradientDescentOptimizer(0.55) 和train = optimizer.minimize(z)，问题似乎就解决了

此代码有效：

import tensorflow as tf

x = tf.Variable(11, name='x', dtype=tf.float32)
y = tf.Variable(11, name='x', dtype=tf.float32)
const = tf.constant(2.0, dtype=tf.float32)

z = x**2 + y**2 + const


optimizer = tf.train.GradientDescentOptimizer(0.55)
train = optimizer.minimize(z)

init = tf.global_variables_initializer()

def optimize():
  with tf.Session() as session:
    session.run(init)
    print("starting at", "x:", session.run(x), "y:", session.run(y), "z:", session.run(z))
    for step in range(10):  
      session.run(train)
      print("step", step, "x:", session.run(x), "y:", session.run(y), "z:", session.run(z))


optimize()

但我想在问题中指定渐变。又名我正在尝试这样做：

def function_to_minimize(x,y, const):
    # z = 2x^2 + y^2 + constant
    z = 2*x**2 + y**2 + const
    return z

def calc_grad(x,y):
    # z = 2x^2 + y^2 + constant
    dz_dx = 4*x
    dz_dy = 2*y
    return [(dz_dx, x), (dz_dy, y)]

x = tf.Variable(3, name='x', dtype=tf.float32)
y = tf.Variable(3, name='y', dtype=tf.float32)
const = tf.constant(2.0, dtype=tf.float32)


z = function_to_minimize(x,y, const)
grad = calc_grad(x,y)


init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
print(sess.run(z))
print(sess.run(grad))


optimizer = tf.train.GradientDescentOptimizer(0.5)

grads_and_vars = calc_grad(x,y)

optimizer.apply_gradients(grads_and_vars)

# minimize() takes care of both computing the gradients and applying them to the variables.
#If you want to process the gradients before applying them you can instead use the optimizer in three steps:
#     1. Compute the gradients with compute_gradients().
#     2. Process the gradients as you wish.
#     3. Apply the processed gradients with apply_gradients()

你如何正确地做到这一点？

【问题讨论】：

标签： python tensorflow machine-learning optimization

【解决方案1】：

apply_gradients 返回一个可用于应用渐变的操作。换句话说，您只需执行train = optimizer.apply_gradients(grads_and_vars)，其余的将与第一个 sn-p 一样工作。我，例如：

optimizer = tf.train.GradientDescentOptimizer(0.55)
grads_and_vars = calc_grad(x,y)
train = optimizer.apply_gradients(grads_and_vars)

init = tf.global_variables_initializer()

def optimize():
  with tf.Session() as session:
    session.run(init)
    print("starting at", "x:", session.run(x), "y:", session.run(y), "z:", session.run(z))
    for step in range(10):  
      session.run(train)
      print("step", step, "x:", session.run(x), "y:", session.run(y), "z:", session.run(z))


optimize()

【讨论】：