在具有公共列的目录中按列合并数据框

Question

我正在尝试将多个 .txt 文件合并到一个目录中，该目录专门合并到数据框中的公共列 X 之一。

import pandas as pd

  df1 = pd.DataFrame({'X': ['X0', 'X1', 'X2', 'X3'],
       ...:                     'B': ['B0', 'B1', 'B2', 'B3'],
       ...:                     'C': ['C0', 'C1', 'C2', 'C3'],
       ...:                     'D': ['D0', 'D1', 'D2', 'D3']})


 df2 = pd.DataFrame({'X': ['X0', 'X1', 'X2', 'X3'],
   ...:                     'G': ['G0', 'G1', 'G2', 'G3'],
   ...:                     'H': ['H0', 'H1', 'H2', 'H3'],
   ...:                     'J': ['J0', 'J1', 'J2', 'J3']})

通过关注之前的这篇文章 Pandas Merging 101 还有这个https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html

我试过了

data = pd.DataFrame()
for file in glob.iglob(filepath + '/*.txt', recursive=True):

    print(file)

    df=pd.read_csv(file, sep ='\t',header=0, skiprows=0, skipfooter=0, na_values=(""," ","NA"))
    data=pd.concat([data,df])

data_merge = pd.merge(data,on='X',how='inner')

但我得到了

TypeError: merge() 缺少 1 个必需的位置参数：'right'

如何通过共同的X 列加入两个数据名？

预期输出

谢谢。

Answer 1

concat 就够了，你不需要merge。执行data = []，将文件读入数据帧，将它们附加到data 和concat 一次：

data = []
for file in glob.iglob(filepath + '/*.txt', recursive=True):

    print(file)

    df=pd.read_csv(file, sep ='\t',header=0, skiprows=0, skipfooter=0, na_values=(""," ","NA"))
    data.append(df.set_index('X'))

data_merge = pd.concat(data, axis=1)

这假定您的 X 列在所有数据框中都不包含重复项。否则，您需要在没有concat 的情况下在for 循环内执行迭代merge，类似于：

final_df = None
for file in glob.iglob(filepath + '/*.txt', recursive=True):

    print(file)

    df=pd.read_csv(file, sep ='\t',header=0, skiprows=0, skipfooter=0, na_values=(""," ","NA"))
    if final_df is not None:
        final_df = final_df.merge(df, on='X', how='outer')
    else: 
        final_df = df

Answer 2

您没有包含要合并的正确数据框，因为您只在合并调用需要两个数据框时才给它一个数据框。尝试： pd.merge(df1, df2, on="X")

编辑：

dataframe_list = []
for file in glob.iglob(filepath + '/*.txt', recursive=True):

    print(file)

    data=pd.read_csv(file, sep ='\t',header=0, skiprows=0, skipfooter=0, na_values=(""," ","NA"))
    dataframe_list.append(data)

data_merge = pd.merge(dataframe_list[0], dataframe_list[1], on='X',how='inner')

---

编辑2，多于两个数据框：

dataframe_list = []
for file in glob.iglob(filepath + '/*.txt', recursive=True):

    print(file)

    data=pd.read_csv(file, sep ='\t',header=0, skiprows=0, skipfooter=0, na_values=(""," ","NA"))
    dataframe_list.append(data)

    if len(dataframe_list) ==2:
        data_merge = pd.merge(dataframe_list[0], dataframe_list[1], on='X')
        dataframe_list = [data_merge]
dataframe = dataframe_list[0]