在tensorflow函数中使用python列表和numpy数组（使用@tf.function注解）答案

【问题标题】：Using python lists and numpy arrays in tensorflow function(using @tf.function annotation)在tensorflow函数中使用python列表和numpy数组（使用@tf.function注解）
【发布时间】：2020-10-15 17:11:28
【问题描述】：

我正在构建一个 LSTM 模型并使用 tensorflow 构建自定义训练循环，以便我可以训练具有不同序列长度的 LSTM 网络。

训练循环正在运行，但与 keras.fitgenerator 相比确实很慢。所以我参考了这些链接：https://github.com/tensorflow/tensorflow/issues/30596，Speeding up Tensorflow 2.0 Gradient Tape。

所以对于我的训练函数，我使用了@tf.function 装饰器，但是在计算损失时我遇到了问题。下面是代码sn-p

import numpy as np
import tensorflow as tf

print("tensorflow version:",tf.__version__)
print("numpy version:",np.__version__)
print("python version:",sys.version)

tensorflow version: 2.1.0
numpy version: 1.18.5
python version: 3.6.7 |Anaconda, Inc.| (default, Oct 28 2018, 19:44:12) [MSC v.1915 64 bit (AMD64)]

# Test data
test_data_ft=[[[1]*23,[2]*23],[[[3]*23,[4]*23,[5]*23]]]
test_data_lb=[[0,1,0],[0,0,1]]
# Network
optimizer_lstmnet=tf.keras.optimizers.Adam()
loss_lstmnet = tf.keras.losses.CategoricalCrossentropy
lstm_net =tf.keras.Sequential()
lstm_net.add(tf.keras.layers.LSTM(32,return_sequences=False,input_shape=(None,23)))
lstm_net.add(tf.keras.layers.Dense(3))
lstm_net.add(tf.keras.layers.Activation('softmax'))
#Training function using tensorflow function
@tf.function
def step(data_ft,data_lb):
    with tf.GradientTape() as tape:
        for i in range(len(data_ft)):
            # This loop makes a forward pass for each sequence
            # reshape the input data. 23 is the feature size, -1 this is the sequence length, we keep it as it is, 1 is the batch size
            x=np.array(data_ft[i]).reshape(1,-1,23)
            x_tens = tf.convert_to_tensor(x,dtype=tf.float32)
            #Make the forward pass
            pred_y = (lstm_net(x_tens))
            # reshape the target label into size (1,3) where 3 is the number of classes
            y= np.array(data_lb[i]).reshape(1,3)
            y_tens=tf.convert_to_tensor(y,dtype=tf.float32)
            # Calculate the loss
            lstm_net_loss += tf.keras.losses.categorical_crossentropy(y_tens, pred_y)
                
        # Calculate loss gradient per batch
        lstm_gradients = tape.gradient((lstm_net_loss), lstm_net.trainable_variables)
        optimizer_lstmnet.apply_gradients(zip(lstm_gradients, lstm_net.trainable_variables))
        return(lstm_net_loss)
#Training loop
for i in range(5):#epochs
    step(test_data_ft,test_data_lb)

这是错误：

 TypeError: in converted code:

    <ipython-input-10-a4181119dd69>:15 step  *
        lstm_net_loss += tf.keras.losses.categorical_crossentropy(y_tens, pred_y)
    C:\Users\rgv1cob\AppData\Local\Continuum\anaconda3\envs\tf_2.2.0\lib\site-packages\tensorflow_core\python\ops\math_ops.py:927 r_binary_op_wrapper
        x = ops.convert_to_tensor(x, dtype=y.dtype.base_dtype, name="x")
    C:\Users\rgv1cob\AppData\Local\Continuum\anaconda3\envs\tf_2.2.0\lib\site-packages\tensorflow_core\python\framework\ops.py:1314 convert_to_tensor
        ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
    C:\Users\rgv1cob\AppData\Local\Continuum\anaconda3\envs\tf_2.2.0\lib\site-packages\tensorflow_core\python\framework\constant_op.py:317 _constant_tensor_conversion_function
        return constant(v, dtype=dtype, name=name)
    C:\Users\rgv1cob\AppData\Local\Continuum\anaconda3\envs\tf_2.2.0\lib\site-packages\tensorflow_core\python\framework\constant_op.py:258 constant
        allow_broadcast=True)
    C:\Users\rgv1cob\AppData\Local\Continuum\anaconda3\envs\tf_2.2.0\lib\site-packages\tensorflow_core\python\framework\constant_op.py:296 _constant_impl
        allow_broadcast=allow_broadcast))
    C:\Users\rgv1cob\AppData\Local\Continuum\anaconda3\envs\tf_2.2.0\lib\site-packages\tensorflow_core\python\framework\tensor_util.py:451 make_tensor_proto
        _AssertCompatible(values, dtype)
    C:\Users\rgv1cob\AppData\Local\Continuum\anaconda3\envs\tf_2.2.0\lib\site-packages\tensorflow_core\python\framework\tensor_util.py:331 _AssertCompatible
        (dtype.name, repr(mismatch), type(mismatch).__name__))

    TypeError: Expected float32, got <tensorflow.python.autograph.operators.special_values.Undefined object at 0x000002D692C857B8> of type 'Undefined' instead.

任何帮助都会很棒。注意：请忽略我的环境名称 tf_2.2.0，我使用的是 tensorflow 2.1.0

【问题讨论】：

标签： python numpy deep-learning tensorflow2.0 tf.keras

【解决方案1】：

我想运行代码来重现该错误。但我发现了一些错误：所以你没有定义值：

lstm_pbu_loss
lstm_pbu

这可能是您的错误的原因？

最好的问候

【讨论】：

非常感谢您的提示。我现在已经更正了代码。但错误是一样的。从错误代码中可以看出问题出在计算损失：:15 step * lstm_net_loss += tf.keras.losses.categorical_crossentropy(y_tens, pred_y)

【解决方案2】：

看起来我必须在使用它之前初始化损失，所以这个更正成功了

def step(data_ft,data_lb):
    lstm_net_loss=0 # The initialization of the loss has to be there..
    ......

任何有关此行为发生原因的信息都会有所帮助

【讨论】：