当激活步骤在函数中时，张量流停止优化

Question

我正在尝试重现 https://www.tensorflow.org/tutorials/mnist/beginners/ 的结果

所以我设计了几个函数来处理训练步骤，比如这两个：

def layer_computation(previous_layer_output, weights, bias, activation):
    return activation(tf.add(tf.matmul(previous_layer_output, weights), bias))

def multilayer_perceptron_forward(x, weights, biaises, activations):
    return reduce(lambda output_layer, args: layer_computation(output_layer, *args),
              zip(weights, biaises, activations), x)

通过使用这两个函数进行训练

def training(session,
         features, labels,
         mlp,
         # cost = (tf.reduce_mean, ),
         optimizer=tf.train.GradientDescentOptimizer,
         epochs=100, learning_rate=0.001, display=100):

x = tf.placeholder("float")
y = tf.placeholder("float")
weights, biases, activations = mlp
pred = multilayer_perceptron_forward(x, weights, biases, activations)
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))
opti = optimizer(learning_rate).minimize(cost)

init = tf.global_variables_initializer()
session.run(init)
for i in range(1, epochs + 1):
    batch_size = 100
    avg_cost = 0
    number_of_bacth = int(features.shape[0]/batch_size)
    for j in range(number_of_bacth):
        my_x = features[j*100:(j+1)*100, :]
        my_y = labels[j*100:(j+1)*100, :]
        _, c = session.run([opti, cost], feed_dict={x: my_x,
                                                    y: my_y})
        avg_cost += c/number_of_bacth

    if i % display == 0:
        print("Epoch {i} cost = {cost}".format(i=i, cost=avg_cost))

优化停止的代价为 2.3...，整体准确度为 10%，而在示例中它接近于零，准确度接近 96%。有人对这种特殊行为有解释吗？

PS 当我在示例的源代码中使用 layer_computation 时，我也会以 2.3 的成本卡住。

Answer 1

我发现了错误，我试图在最后一层执行反向传播。这个问题在交叉验证时可能会更好。