Keras 输入 Pandas 数据框

Question

我是 Keras 的新手，我想将我的火车数据放入 Excel 文件中。 我的数据有 shape(1000, 5, 5)，1000 批数据保存在 1000 个电子表格中， 每张纸包含 5 列和 5 行：

A	B	C	D	E
-	-	-	-	label
-	-	-	-	label
-	-	-	-	label
-	-	-	-	label
-	-	-	-	label

我希望 A、B、C 列作为训练特征，E 列作为标签。

import pandas as pd
import tensorflow as tf
import multiprocessing

df = pd.read_excel('File.xlsx', sheet_name=None)
data_list = list(df.values())

def input_parser(x):
    Y = x.pop('E')
    features = ['A','B','C']
    X = x[features]
    return X, Y

dataset = tf.data.Dataset.from_tensor_slices(data_list)
dataset = dataset.map(lambda x: tuple(tf.py_function(func=input_parser,
                                                     inp=[x],
                                                     Tout=[tf.float32,tf.int64])),
                      num_parallel_calls=multiprocessing.cpu_count())

然后我得到一个错误：

ValueError: Can't convert non-rectangular Python sequence to Tensor.

为什么会出现此错误？ 如何将这些数据拟合到我的模型中？

Answer 1

也许可以尝试完全省略您的 map 函数，只需将您的数据直接传递给 tf.data.Dataset.from_tensor_slices：

import pandas as pd
import tensorflow as tf
import numpy as np

spread_sheet1 = {'A': [1, 2, 1, 2, 9], 'B': [3, 4, 6, 1, 4], 'C': [3, 4, 3, 1, 4], 'D': [1, 2, 6, 1, 4], 'E': [0, 1, 1, 0, 1]}
df1 = pd.DataFrame(data=spread_sheet1)

spread_sheet2 = {'A': [1, 2, 1, 2, 4], 'B': [3, 5, 2, 1, 4], 'C': [9, 4, 1, 1, 4], 'D': [1, 5, 6, 1, 7], 'E': [1, 1, 1, 0, 1]}
df2 = pd.DataFrame(data=spread_sheet2)

features = ['A','B','C']
Y = np.stack([df1['E'].to_numpy(), df2['E'].to_numpy()])
Y = tf.convert_to_tensor(Y, dtype=tf.int32)
X = np.stack([df1[features].to_numpy(), df2[features].to_numpy()])
X = tf.convert_to_tensor(X, dtype=tf.float32)


dataset = tf.data.Dataset.from_tensor_slices((X, Y))
print('Shape of X --> ', X.shape)
for x, y in dataset:
  print(x, y)

Shape of X -->  (2, 5, 3)
tf.Tensor(
[[1. 3. 3.]
 [2. 4. 4.]
 [1. 6. 3.]
 [2. 1. 1.]
 [9. 4. 4.]], shape=(5, 3), dtype=float32) tf.Tensor([0 1 1 0 1], shape=(5,), dtype=int32)
tf.Tensor(
[[1. 3. 9.]
 [2. 5. 4.]
 [1. 2. 1.]
 [2. 1. 1.]
 [4. 4. 4.]], shape=(5, 3), dtype=float32) tf.Tensor([1 1 1 0 1], shape=(5,), dtype=int32)

从带有多个工作表的 excel 文件file.xlsx 中读取可以这样完成：

import pandas as pd
import tensorflow as tf
import multiprocessing

df = pd.read_excel('file.xlsx', sheet_name=None)
file_names = list(df.keys())

columns = ['A','B','C']
features = []
labels = []
for n in file_names:
  temp_df = df[n]
  features.append(temp_df[columns].to_numpy())
  labels.append(temp_df['E'].to_numpy())
  
Y = tf.convert_to_tensor(np.stack(labels), dtype=tf.int32)
X = tf.convert_to_tensor(np.stack(features), dtype=tf.float32)
dataset = tf.data.Dataset.from_tensor_slices((X, Y))

print('Shape of X --> ', X.shape)
for x, y in dataset:
  print(x, y)