【问题标题】:Python error in calculating deep learning model value loss - "ValueError: Data Cardinality is ambiguous"Python 计算深度学习模型价值损失时出错 - “ValueError: Data Cardinality is ambiguous”
【发布时间】:2020-08-20 16:51:20
【问题描述】:

尽管之前有 R 和 Python 的编码经验,但我对 Jupyter Notebook、tensorflow 和深度学习模型构建是全新的,所以我正在寻找帮助我诊断此错误的人。我正在学习一个演示如何使用深度学习模块对图像进行分类的教程 (https://www.youtube.com/watch?v=wQ8BIBpya2k&list=PLQVvvaa0QuDfhTox0AjmQ6tvTgMBZBEXN)。模型加载 mnist 图像数据集,并将图像分类在 1-10 之间。运行三个 epoch,整体模型准确率达到 97%。

#Import module
import tensorflow as tf

#Import image dataset
mnist = tf.keras.datasets.mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()

#Defining training and test
x_train = tf.keras.utils.normalize(x_train, axis = 1)
x_test = tf.keras.utils.normalize(x_train, axis = 1)

#Define model
model = tf.keras.models.Sequential() #Feed-forward model

#Define imput layer
model.add(tf.keras.layers.Flatten()) #Input layer

#Two hidden layers
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu)) #Rectify linear, default
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))

#Output layer
#Corresponds to the number of classifications; ten in this case
#No relu because it is a probability distribution
model.add(tf.keras.layers.Dense(10, activation=tf.nn.softmax))

#Defining training parameters for the model
#Loss is error, or what you have gotten wrong
model.compile(optimizer = 'adam',
             loss= 'sparse_categorical_crossentropy', #could use binary if looking for cats/dogs
             metrics = ['accuracy'])

#Training the model
#Epoch = how many times the model runs
model.fit(x_train, y_train, epochs = 3)

输出如下:

Epoch 1/3
1875/1875 [==============================] - 1s 578us/step - loss: 0.2612 - accuracy: 0.9236
Epoch 2/3
1875/1875 [==============================] - 1s 571us/step - loss: 0.1068 - accuracy: 0.9668
Epoch 3/3
1875/1875 [==============================] - 1s 562us/step - loss: 0.0721 - accuracy: 0.9773

当我试图找出模型损失时......

#Printing the model loss
val_loss, val_acc = model.evaluate(x_test, y_test)
print(val_loss, val_acc)

...我收到此错误:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-3452f7a38776> in <module>
----> 1 val_loss, val_acc = model.evaluate(x_test, y_test)
      2 print(val_loss, val_acc)

~\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\keras\engine\training.py in _method_wrapper(self, *args, **kwargs)
    106   def _method_wrapper(self, *args, **kwargs):
    107     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
--> 108       return method(self, *args, **kwargs)
    109 
    110     # Running inside `run_distribute_coordinator` already.

~\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\keras\engine\training.py in evaluate(self, x, y, batch_size, verbose, sample_weight, steps, callbacks, max_queue_size, workers, use_multiprocessing, return_dict)
   1354             use_multiprocessing=use_multiprocessing,
   1355             model=self,
-> 1356             steps_per_execution=self._steps_per_execution)
   1357 
   1358       # Container that configures and calls `tf.keras.Callback`s.

~\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\keras\engine\data_adapter.py in __init__(self, x, y, sample_weight, batch_size, steps_per_epoch, initial_epoch, epochs, shuffle, class_weight, max_queue_size, workers, use_multiprocessing, model, steps_per_execution)
   1115         use_multiprocessing=use_multiprocessing,
   1116         distribution_strategy=ds_context.get_strategy(),
-> 1117         model=model)
   1118 
   1119     strategy = ds_context.get_strategy()

~\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\keras\engine\data_adapter.py in __init__(self, x, y, sample_weights, sample_weight_modes, batch_size, epochs, steps, shuffle, **kwargs)
    280             label, ", ".join(str(i.shape[0]) for i in nest.flatten(data)))
    281       msg += "Please provide data which shares the same first dimension."
--> 282       raise ValueError(msg)
    283     num_samples = num_samples.pop()
    284 

ValueError: Data cardinality is ambiguous:
  x sizes: 60000
  y sizes: 10000
Please provide data which shares the same first dimension.

我可以做些什么来修复这个错误并计算损失?

【问题讨论】:

    标签: python tensorflow keras deep-learning jupyter-notebook


    【解决方案1】:

    正如错误所说,x_test 中对应于y_test 的样本数不匹配

      x sizes: 60000
      y sizes: 10000
    

    如果您检查您的代码,您会发现在创建 x_test 时存在错误

    应该是:

    x_test = tf.keras.utils.normalize(x_test, axis = 1)
    

    不是

    x_test = tf.keras.utils.normalize(x_train, axis = 1)
    

    【讨论】:

      猜你喜欢
      • 2021-02-10
      • 2020-12-21
      • 2020-11-10
      • 1970-01-01
      • 2018-12-27
      • 2020-11-09
      • 2019-04-04
      • 1970-01-01
      • 2023-03-29
      相关资源
      最近更新 更多