update_state() 和 call() 方法多久调用一次，tf.keras 中参数的维度是多少？

Question

我想知道

(1) tf.keras.losses.Loss 的 call() 方法多久调用一次 并且在训练期间调用 tf.keras.metrics.Metric 的 update_state() 方法：

• 每个实例都调用它们（观察）吗？
• 还是按批次调用？

(2) y_true 和 y_pred 的维度传递给那些方法：

• 是它们的尺寸（batch_size x output_dimension）
• 或（1 x output_dimension）

以下代码sn-p来自 https://www.tensorflow.org/guide/keras/train_and_evaluate

对于实验，我在 update_state() 中插入 print(y_true.shape, y_pred.shape) ，我发现它在第一个时期只打印一次。从打印上看，y_true 和 y_pred 的尺寸为 (1 x output_dimension) 在这个特定的例子中，但总是这样吗？

所以，另外

(3) 我想知道为什么它只在第一个 epoch 打印一次。

(4) 我无法打印 y_true 或 y_pred 的值。我该怎么办？

Epoch 1/3
(None, 1) (None, 10)
(None, 1) (None, 10)
782/782 [==============================] - 3s 4ms/step - loss: 0.5666 - categorical_true_positives: 22080.8940
Epoch 2/3
782/782 [==============================] - 3s 4ms/step - loss: 0.1680 - categorical_true_positives: 23877.1162
Epoch 3/3
782/782 [==============================] - 3s 4ms/step - loss: 0.1190 - categorical_true_positives: 24198.2733
<tensorflow.python.keras.callbacks.History at 0x1fb132cde80>

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

# Preprocess the data (these are NumPy arrays)
x_train = x_train.reshape(60000, 784).astype("float32") / 255
x_test = x_test.reshape(10000, 784).astype("float32") / 255

y_train = y_train.astype("float32")
y_test = y_test.astype("float32")

# Reserve 10,000 samples for validation
x_val = x_train[-10000:]
y_val = y_train[-10000:]
x_train = x_train[:-10000]
y_train = y_train[:-10000]

inputs = keras.Input(shape=(784,), name="digits")
x = layers.Dense(64, activation="relu", name="dense_1")(inputs)
x = layers.Dense(64, activation="relu", name="dense_2")(x)
outputs = layers.Dense(10, activation="softmax", name="predictions")(x)

model = keras.Model(inputs=inputs, outputs=outputs)

class CategoricalTruePositives(keras.metrics.Metric):
    def __init__(self, name="categorical_true_positives", **kwargs):
        super(CategoricalTruePositives, self).__init__(name=name, **kwargs)
        self.true_positives = self.add_weight(name="ctp", initializer="zeros")

    def update_state(self, y_true, y_pred, sample_weight=None):
        print(y_true.shape, y_pred.shape) # For experiment
        y_pred = tf.reshape(tf.argmax(y_pred, axis=1), shape=(-1, 1))
        values = tf.cast(y_true, "int32") == tf.cast(y_pred, "int32")
        values = tf.cast(values, "float32")
        if sample_weight is not None:
            sample_weight = tf.cast(sample_weight, "float32")
            values = tf.multiply(values, sample_weight)
        self.true_positives.assign_add(tf.reduce_sum(values))

    def result(self):
        return self.true_positives

    def reset_states(self):
        # The state of the metric will be reset at the start of each epoch.
        self.true_positives.assign(0.0)
        
model.compile(
    optimizer=keras.optimizers.RMSprop(learning_rate=1e-3),
    loss=keras.losses.SparseCategoricalCrossentropy(),
    metrics=[CategoricalTruePositives()],
)
model.fit(x_train, y_train, batch_size=64, epochs=3)   

Answer 1

(1) tf.keras.losses.Loss 的 call() 方法和 tf.keras.metrics.Metric 的 update_state() 方法在训练期间被调用的频率：

tf.keras.losses.Loss和update_state()的调用方法在每批结束时使用。

(2) y_true 和 y_pred 的维度传递给那些方法：

y_true 的尺寸与您在y_train 中传递的尺寸相同。唯一的变化是，y_train 的第一个维度将是 no_of samples，而对于 y_true，它将是 batch_size。在您的情况下，它是 (64, 1)，其中 64 是 batch_size。

y_pred 的维度是模型输出的形状。在您的情况下，它是 (64, 10) 因为您在最后一层有 10 个密集单元。

(3) 我想知道为什么它只在第一个 epoch 打印一次。

print 语句只执行一次，因为 tensorflow 是在图形模式下执行的。打印只能在急切模式下工作。如果要在 Eager 模式下执行 TensorFlow 代码，请在 model.compile 步骤中添加 run_eagerly = True。

(4) 我无法打印 y_true 或 y_pred 的值。我该怎么办？

以 Eager 模式运行代码。

代码：

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

# Preprocess the data (these are NumPy arrays)
x_train = x_train.reshape(60000, 784).astype("float32") / 255
x_test = x_test.reshape(10000, 784).astype("float32") / 255

y_train = y_train.astype("float32")
y_test = y_test.astype("float32")

# Reserve 10,000 samples for validation
x_val = x_train[-10000:]
y_val = y_train[-10000:]
x_train = x_train[:-10000]
y_train = y_train[:-10000]

inputs = keras.Input(shape=(784,), name="digits")
x = layers.Dense(64, activation="relu", name="dense_1")(inputs)
x = layers.Dense(64, activation="relu", name="dense_2")(x)
outputs = layers.Dense(10, activation="softmax", name="predictions")(x)

model = keras.Model(inputs=inputs, outputs=outputs)

class CategoricalTruePositives(keras.metrics.Metric):
    def __init__(self, name="categorical_true_positives", **kwargs):
        super(CategoricalTruePositives, self).__init__(name=name, **kwargs)
        self.true_positives = self.add_weight(name="ctp", initializer="zeros")

    def update_state(self, y_true, y_pred, sample_weight=None):
        print('update_state', y_true.shape, y_pred.shape) # For experiment
        y_pred = tf.reshape(tf.argmax(y_pred, axis=1), shape=(-1, 1))
        values = tf.cast(y_true, "int32") == tf.cast(y_pred, "int32")
        values = tf.cast(values, "float32")
        if sample_weight is not None:
            sample_weight = tf.cast(sample_weight, "float32")
            values = tf.multiply(values, sample_weight)
        self.true_positives.assign_add(tf.reduce_sum(values))

    def result(self):
        return self.true_positives

    def reset_states(self):
        # The state of the metric will be reset at the start of each epoch.
        self.true_positives.assign(0.0)

class CustomCallback(tf.keras.callbacks.Callback):
  def on_epoch_begin(self, epoch, logs=None):
    print("Start epoch {} of training".format(epoch))
  def on_train_batch_begin(self, batch, logs=None):
    keys = list(logs.keys())
    print("...Training: start of batch {}".format(batch))
  def on_train_batch_end(self, batch, logs=None):
    print("...Training: end of batch {}".format(batch))
        
model.compile(
    optimizer=keras.optimizers.RMSprop(learning_rate=1e-3),
    loss=keras.losses.SparseCategoricalCrossentropy(),
    metrics=[CategoricalTruePositives()],
    run_eagerly = True,
)
model.fit(x_train, y_train, batch_size=64, epochs=3, verbose = 0, callbacks=[CustomCallback()])

输出：

Start epoch 0 of training
...Training: start of batch 0
update_state (64, 1) (64, 10)
...Training: end of batch 0
...Training: start of batch 1
update_state (64, 1) (64, 10)
...Training: end of batch 1
...Training: start of batch 2
update_state (64, 1) (64, 10)
...Training: end of batch 2
...Training: start of batch 3
update_state (64, 1) (64, 10)
...Training: end of batch 3
...Training: start of batch 4
update_state (64, 1) (64, 10)
...Training: end of batch 4
...Training: start of batch 5
update_state (64, 1) (64, 10)
...Training: end of batch 5

上面的例子会让你的答案一目了然。