无法重新索引重复轴

Question

我正在尝试将多个 csv 文件合并到一个文件夹中。

它们看起来像这样（实际上有两个以上的 df）：

df1

LCC  acres
2    10
3    20
4    40
5    5

df2

LCC  acres_2
2    4
3    2
4    40
5    6
6    7

我想将所有数据框放入一个列表中，然后将它们与 reduce 合并。为此，它们需要具有相同的索引。

我正在尝试这段代码：

combined = []
reindex = [2,3,4,5,6]

folder = r'C:\path_to_files'

for f in os.listdir(folder):

    #read each file
    df = pd.read_csv(os.path.join(folder,f))

    #check for duplicates - returns empty lists
    print df[df.index.duplicated()]

    #reindex
    df.set_index([df.columns[0]], inplace=True)
    df=df.reindex(reindex, fill_value=0)

    #append
    combined.append(df)


#merge on 'LCC' column
final = reduce(lambda left, right: pd.merge(left, right, on=['LCC'], how='outer'), combined)

但这仍然返回：

Traceback (most recent call last):

  File "<ipython-input-31-45f925f6d48d>", line 9, in <module>
    df=df.reindex(reindex, fill_value=0)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\frame.py", line 2741, in reindex
    **kwargs)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\generic.py", line 2229, in reindex
    fill_value, copy).__finalize__(self)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\frame.py", line 2687, in _reindex_axes
    fill_value, limit, tolerance)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\frame.py", line 2698, in _reindex_index
    allow_dups=False)

File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\generic.py", line 2341, in _reindex_with_indexers
    copy=copy)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\internals.py", line 3586, in reindex_indexer
    self.axes[axis]._can_reindex(indexer)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\indexes\base.py", line 2293, in _can_reindex
    raise ValueError("cannot reindex from a duplicate axis")

ValueError: cannot reindex from a duplicate axis

Answer 1

在将第一列设置为索引后，您需要检查索引的重复项。

#set index by first column
df.set_index([df.columns[0]], inplace=True)

#check for duplicates - returns NO empty lists
print df[df.index.duplicated()]

#reindex
df=df.reindex(reindex, fill_value=0)

或者检查第一列中的重复而不是索引，参数keep=False返回所有重复（如果需要）：

#check duplicates in first column
print df[df.iloc[:, 0].duplicated(keep=False)]

#set index + reindex
df.set_index([df.columns[0]], inplace=True)
df=df.reindex(reindex, fill_value=0)