ValueError：即使在 np.concatenate 中使用元组后，也无法连接零维数组答案

【问题标题】：ValueError: zero-dimensional arrays cannot be concatenated even after using tuple in np.concatenateValueError：即使在 np.concatenate 中使用元组后，也无法连接零维数组
【发布时间】：2020-08-15 09:48:06
【问题描述】：

我正在尝试应用一个工作脚本（input-2 上的code2），它将数据框字符串条目转换为我的数据集（output-1）上看起来类似于input-2 的单独行。奇怪的是，我得到了 ValueError: zero-dimensional arrays 错误。我用谷歌搜索并搜索了 StackOverflow，但没有成功。

错误

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-310-36a8923ddf69> in <module>
----> 1 explode(df3, ['longitude','altitude', 'speed'], fill_value='',preserve_index=True)

<ipython-input-308-31db0d7d3bc1> in explode(df, lst_cols, fill_value, preserve_index)
     19                 index=idx)
     20              .assign(**{col:np.concatenate(df.loc[lens>0, col].values)
---> 21                             for col in lst_cols}))
     22     # append those rows that have empty lists
     23     if (lens == 0).any():

<ipython-input-308-31db0d7d3bc1> in <dictcomp>(.0)
     19                 index=idx)
     20              .assign(**{col:np.concatenate(df.loc[lens>0, col].values)
---> 21                             for col in lst_cols}))
     22     # append those rows that have empty lists
     23     if (lens == 0).any():

ValueError: zero-dimensional arrays cannot be concatenated

Json 文件 (soverflowq.json)

{'longitude': [24.64977040886879, 24.65014273300767, 24.6501], 'altitude': [41.2, 34.2, 24.6501],'url': 'https://www.endomondo.com/users/10921916/workouts/392337037', 'userId': 10921916, 'speed': [9.0792, 13.284, 24.6501]}
{'longitude': [22.44977040886879, 27.65014273300767, 24.6501], 'altitude': [38.4, 39.0, 24.6501],'url': 'https://www.endomondo.com/users/10921915/workouts/392337038', 'userId': 10921915, 'speed': [9.0792, 13.284, 24.6501]}
{'longitude': [24.64977040886879, 24.65014273300767, 24.6501], 'altitude': [41.2, 34.2, 24.6501],'url': 'https://www.endomondo.com/users/10921916/workouts/392337037', 'userId': 1092116, 'speed': [9.0792, 13.284, 24.6501]}
{'longitude': [22.44977040886879, 27.65014273300767, 24.6501], 'altitude': [38.4, 39.0, 24.6501],'url': 'https://www.endomondo.com/users/10921915/workouts/392337038', 'userId': 1092191, 'speed': [9.0792, 13.284, 24.6501]}

代码 1

import pandas as pd
import re

# read a file line-by-line into a list
with open('soverflowq.json') as f:
    data = f.readlines()

# split each line into one column
data = [re.split(r'\s+(?=\d+$)', l) for l in data]  
# constructing dataframe
df = pd.DataFrame(data, columns=['all_cols'])   
# split further and rename the cols
df[['new0','new1', 'new2', 'new3', 'new4', 'new5', 'new6', 'new7', 'new8', 'new9']] = df['all_cols'].str.split(r"\, '|\,'|\':",expand=True) 

# remove [, ], ', }, \n
cols_to_check = ['new0','new1','new3','new5', 'new9'] 
df[cols_to_check] = df[cols_to_check].replace({'\'':''}, regex=True)
df[cols_to_check] = df[cols_to_check].replace({'{':''}, regex=True)
df[cols_to_check] = df[cols_to_check].replace({'}':''}, regex=True)
df[cols_to_check] = df[cols_to_check].replace({'\n':''}, regex=True)
df[cols_to_check] = df[cols_to_check].replace({'\n':''}, regex=True)
#df[cols_to_check] = df[cols_to_check].replace({'\[':''}, regex=True)
#df[cols_to_check] = df[cols_to_check].replace({'\]':''}, regex=True)

#drop cols
df1= df.drop(columns=['all_cols', 'new0', 'new2', 'new4', 'new6', 'new8'])

# rename cols
df1.columns = ['longitude', 'altitude', 'url', 'userId', 'speed']

# reorder cols
column_titles =['userId', 'longitude', 'altitude', 'speed', 'url']
df2 = df1.reindex(columns=column_titles)
df3= df2.drop(columns=['url'])
print(df3)

输出 1

      userId                                         longitude  \
0   10921916   [24.64977040886879, 24.65014273300767, 24.6501]   
1   10921915   [22.44977040886879, 27.65014273300767, 24.6501]   
2    1092116   [24.64977040886879, 24.65014273300767, 24.6501]   
3    1092191   [22.44977040886879, 27.65014273300767, 24.6501]   

                 altitude                       speed  
0   [41.2, 34.2, 24.6501]   [9.0792, 13.284, 24.6501]  
1   [38.4, 39.0, 24.6501]   [9.0792, 13.284, 24.6501]  
2   [41.2, 34.2, 24.6501]   [9.0792, 13.284, 24.6501]  
3   [38.4, 39.0, 24.6501]   [9.0792, 13.284, 24.6501]

代码 2

Source

def explode(df, lst_cols, fill_value='', preserve_index=False):
    # make sure `lst_cols` is list-alike
    if (lst_cols is not None
        and len(lst_cols) > 0
        and not isinstance(lst_cols, (list, tuple, np.ndarray, pd.Series))):
        lst_cols = [lst_cols]
    # all columns except `lst_cols`
    idx_cols = df.columns.difference(lst_cols)
    # calculate lengths of lists
    lens = df[lst_cols[0]].str.len()
    # preserve original index values    
    idx = np.repeat(df.index.values, lens)
    # create "exploded" DF
    res = (pd.DataFrame({
                col:np.repeat(df[col].values, lens)
                for col in idx_cols},
                index=idx)
             .assign(**{col:np.concatenate((df.loc[lens>0, col].values))
                            for col in lst_cols}))
    # append those rows that have empty lists
    if (lens == 0).any():
        # at least one list in cells is empty
        res = (res.append(df.loc[lens==0, idx_cols], sort=False)
                  .fillna(fill_value))
    # revert the original index order
    res = res.sort_index()
    # reset index if requested
    if not preserve_index:        
        res = res.reset_index(drop=True)
    return res

输入 2

df = pd.DataFrame({
    'userid': {0: 10, 1: 2, 2: 12, 3: 13},
    'longitude': {0: [24.64977040886879, 24.64977040886879, 324.64977040886879], 
            1: [24.64977040886879,424.64977040886879,524.64977040886879], 
            2: [124.64977040886879,224.64977040886879 ,224.64977040886879], 
            3: [124.64977040886879,324.64977040886879,424.64977040886879]},
    'altitude': {0: [24.64977040886879, 24.64977040886879, 324.64977040886879], 
            1: [24.64977040886879,424.64977040886879,524.64977040886879], 
            2: [124.64977040886879,224.64977040886879 ,224.64977040886879], 
            3: [124.64977040886879,324.64977040886879,424.64977040886879]},
    'speed': {0: [24.64977040886879, 24.64977040886879, 324.64977040886879], 
            1: [24.64977040886879,424.64977040886879,524.64977040886879], 
            2: [124.64977040886879,224.64977040886879 ,224.64977040886879], 
            3: [124.64977040886879,324.64977040886879,424.64977040886879]},
})



explode(df, ['longitude','altitude', 'speed'], fill_value='',preserve_index=True)

输出 2

userid  longitude   altitude    speed
0   10  24.64977    24.64977    24.64977
0   10  24.64977    24.64977    24.64977
0   10  324.64977   324.64977   324.64977
1   2   24.64977    24.64977    24.64977
1   2   424.64977   424.64977   424.64977
1   2   524.64977   524.64977   524.64977
2   12  124.64977   124.64977   124.64977
2   12  224.64977   224.64977   224.64977
2   12  224.64977   224.64977   224.64977
3   13  124.64977   124.64977   124.64977
3   13  324.64977   324.64977   324.64977
3   13  424.64977   424.64977   424.64977

【问题讨论】：

标签： python pandas numpy

【解决方案1】：

您不提供回溯或问题表达式中的示例值。但根据标题，我猜问题出在

np.concatenate((df.loc[lens>0, col].values))

表达式。我无法重新创建您的 df，但我怀疑字符串列存在问题（[24.64977040886879, 24.64977040886879, 324.64977040886879] 可能看起来像一个列表，但它很可能是一个字符串。Pandas 在其显示中省略了引号。

作为一项实验，我采用了您的 json 行之一，并“解析”了它：

In [159]: dd = {'longitude': [24.64977040886879, 24.65014273300767, 24.6501], 'altitude': [41.2, 34.2, 
     ...: 24.6501],'url': 'https://www.endomondo.com/users/10921916/workouts/392337037', 'userId': 1092
     ...: 1916, 'speed': [9.0792, 13.284, 24.6501]}                                                    
In [160]: dd                                                                                           
Out[160]: 
{'longitude': [24.64977040886879, 24.65014273300767, 24.6501],
 'altitude': [41.2, 34.2, 24.6501],
 'url': 'https://www.endomondo.com/users/10921916/workouts/392337037',
 'userId': 10921916,
 'speed': [9.0792, 13.284, 24.6501]}

然后从中制作一个数据框：

In [161]: pd.DataFrame(dd)                                                                             
Out[161]: 
   longitude  altitude                                                url    userId    speed
0  24.649770   41.2000  https://www.endomondo.com/users/10921916/worko...  10921916   9.0792
1  24.650143   34.2000  https://www.endomondo.com/users/10921916/worko...  10921916  13.2840
2  24.650100   24.6501  https://www.endomondo.com/users/10921916/worko...  10921916  24.6501
In [162]: _.dtypes                                                                                     
Out[162]: 
longitude    float64
altitude     float64
url           object
userId         int64
speed        float64
dtype: object


In [163]: _161['url']                                                                                  
Out[163]: 
0    https://www.endomondo.com/users/10921916/worko...
1    https://www.endomondo.com/users/10921916/worko...
2    https://www.endomondo.com/users/10921916/worko...
Name: url, dtype: object
In [164]: _161['url'].values                                                                           
Out[164]: 
array(['https://www.endomondo.com/users/10921916/workouts/392337037',
       'https://www.endomondo.com/users/10921916/workouts/392337037',
       'https://www.endomondo.com/users/10921916/workouts/392337037'],
      dtype=object)

所以这个“值”是一个字符串数组（对象 dtype 对于熊猫来说是正常的）。尝试对此执行 concatenate 会产生错误。

In [165]: np.concatenate(_161['url'].values)                                                           
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-165-df206e02c51f> in <module>
----> 1 np.concatenate(_161['url'].values)

<__array_function__ internals> in concatenate(*args, **kwargs)

ValueError: zero-dimensional arrays cannot be concatenated

将“值”包装在 () 中不会改变任何事情。添加逗号确实使它成为一个元组：

In [168]: np.concatenate((_161['url'].values,))                                                        
Out[168]: 
array(['https://www.endomondo.com/users/10921916/workouts/392337037',
       'https://www.endomondo.com/users/10921916/workouts/392337037',
       'https://www.endomondo.com/users/10921916/workouts/392337037'],
      dtype=object)

但是在 1 元素元组上连接并没有做任何重要的事情。

我怀疑您需要检查df.loc[lens>0, col].values，并准确了解它是什么。没有猜测或盲测。

【讨论】：

我添加了 Traceback。我很惊讶地看到您无法重现我的数据框（输出 1）。它适用于我的 Jupyter 笔记本。我创建了一个联合实验室link，以便您对其进行测试。请试试这个并告诉我。
另外，我在运行代码之前从数据框中删除了“url 值”。所以这应该不是问题。
另一种列元素实际上不是列表的情况，stackoverflow.com/q/61531470/901925
那么这应该不起作用但它没有df3[['longitude', 'altitude', 'speed']] = df3[['longitude', 'altitude', 'speed']].values.tolist()