导入熊猫数据框的Python问题

Question

我有一个具有这种结构的 csv 文件：

Path name	Path ID	Phenotype data
path name 1	ID 1	Table 1
path name 2	ID 2	Table 2

最后一个单元格中的每个表格都有这样的结构：

Gene_MGI_ID	Gene_ENSEMBL_ID	Phenotype list
MGI_1	ENS_1	[Pheno1,Pheno2,Pheno 3,...]
MGI_2	ENS_2	[Pheno5,Pheno8,Pheno 9,...]

我正在使用此代码导入文件：

path=pd.read_csv(dir+"path_dataset.csv", header=0, index_col=0)

问题在于最后一列不是包含这 3 列的数据框，而是单个字符串。例如，导入后的第一个表由以下字符串表示：

'      MGI_ID              ENS_ID                                         Pheno_list\n0  MGI:99702  ENSMUSG00000003873  [MP:0000218, MP:0000321, MP:0000571, MP:000069...'

如何在上述数据框中转换此字符串？ 这里以表格的前三个元素作为 csv 文件结构的示例：

,Pathway,Path_ID,Pheno_data
0,response to organophosphorus,GO:0046683,"      MGI_ID              ENS_ID                                         Pheno_list
0  MGI:96648  ENSMUSG00000071076  [MP:0000278, MP:0000351, MP:0001156, MP:000126..."
1,response to cAMP,GO:0051591,"      MGI_ID              ENS_ID                                         Pheno_list
0  MGI:96648  ENSMUSG00000071076  [MP:0000278, MP:0000351, MP:0001156, MP:000126..."
2,secondary metabolite biosynthetic process,GO:0044550,"        MGI_ID              ENS_ID    Pheno_list

Answer 1

pheno_data 看起来像是将数据帧的文本转储到单个单元格中。由于默认格式，它截断了输出，[ 没有结束 ]，最后一个值显示 ...。

您的示例还缺少最后一行的后半部分（没有结束 "）。对于以下示例（添加了最后一行）：

,Pathway,Path_ID,Pheno_data
0,response to organophosphorus,GO:0046683,"      MGI_ID              ENS_ID                                         Pheno_list
0  MGI:96648  ENSMUSG00000071076  [MP:0000278, MP:0000351, MP:0001156, MP:000126..."
1,response to cAMP,GO:0051591,"      MGI_ID              ENS_ID                                         Pheno_list
0  MGI:96648  ENSMUSG00000071076  [MP:0000278, MP:0000351, MP:0001156, MP:000126..."
2,secondary metabolite biosynthetic process,GO:0044550,"        MGI_ID              ENS_ID    Pheno_list
0  MGI:96648  ENSMUSG00000071076  [MP:0000278, MP:0000351, MP:0001156, MP:000126..."

以下脚本可用于读取值：

import pandas as pd
import csv
import io

data = []

with open('input.csv') as f_input:
    csv_input = csv.reader(f_input)
    header = next(csv_input)
    
    for row in csv_input:
        sub_csv = csv.reader(io.StringIO(row[3]), delimiter=' ', skipinitialspace=True)
        sub_header = next(sub_csv)
        sub_row = next(sub_csv)
        pheno_list = [v.strip('[.,') for v in sub_row[3:]]
        data.append([*row[:2], *sub_row[:3], *pheno_list])
        
df = pd.DataFrame(data, columns=[*header[:3], *sub_header[:2], *[f'p{i}' for i in range(1, 5)]])

print(df)

这将给出一个数据框：

                                        Pathway Path_ID     MGI_ID              ENS_ID          p1          p2          p3         p4
0  0               response to organophosphorus       0  MGI:96648  ENSMUSG00000071076  MP:0000278  MP:0000351  MP:0001156  MP:000126
1  1                           response to cAMP       0  MGI:96648  ENSMUSG00000071076  MP:0000278  MP:0000351  MP:0001156  MP:000126
2  2  secondary metabolite biosynthetic process       0  MGI:96648  ENSMUSG00000071076  MP:0000278  MP:0000351  MP:0001156  MP:000126

如果您在读取文件时遇到问题，请尝试添加 print(row) 以查看失败的位置。

---

要将 pheno 类型保留为单个条目，请更改以下两行：

data.append([*row[:2], *sub_row[:3], ', '.join(pheno_list)])
    
df = pd.DataFrame(data, columns=[*header[:3], *sub_header[:3]])