在 python3 的 numpy.txt 中打开 csv 文件

Question

我有一个 csv 文件并尝试使用 numpy.loadtxt 打开它。 如果我使用 pandas 打开它，文件将如下所示：

小例子：

Name       Accession  Class Species  Annotation  CF330
NaN             NaN         NaN          NaN         NaN         NaN
A2M     NM_000014.4  Endogenous           Hs         NaN     11495.0
ACVR1C     NM_145259.2  Endogenous           Hs         NaN        28.0
ADAM12     NM_003474.5  Endogenous           Hs         NaN      1020.0
ADGRE1  NM_001256252.1  Endogenous           Hs         NaN        42.0

我正在尝试使用 numpy.loadtxt 并使用以下代码打开文件：

with open('datafile1.csv') as f:
    for line in f:
        FH = np.loadtxt(line, delimiter=',', skiprows=1)

print(FH)

但它返回此错误：

ValueError: could not convert string to float: 

你知道如何解决这个问题吗？

这里是原始数据集：

Name,Accession,Class,Species,Annotation,CF330
,,,,,
A2M,NM_000014.4,Endogenous,Hs,,11495
ACVR1C,NM_145259.2,Endogenous,Hs,,28
ADAM12,NM_003474.5,Endogenous,Hs,,1020
ADGRE1,NM_001256252.1,Endogenous,Hs,,42

Answer 1

In [19]: txt = '''Name,Accession,Class,Species,Annotation,CF330 
    ...: ,,,,, 
    ...: A2M,NM_000014.4,Endogenous,Hs,,11495 
    ...: ACVR1C,NM_145259.2,Endogenous,Hs,,28 
    ...: ADAM12,NM_003474.5,Endogenous,Hs,,1020 
    ...: ADGRE1,NM_001256252.1,Endogenous,Hs,,42'''   

使用dtype=None，genfromtxt 为我们提供了一个结构化数组：

In [23]: np.genfromtxt(txt.splitlines(), names=True, dtype=None, encoding=None,delimiter=',')        
Out[23]: 
array([('', '', '', '', False,    -1),
       ('A2M', 'NM_000014.4', 'Endogenous', 'Hs', False, 11495),
       ('ACVR1C', 'NM_145259.2', 'Endogenous', 'Hs', False,    28),
       ('ADAM12', 'NM_003474.5', 'Endogenous', 'Hs', False,  1020),
       ('ADGRE1', 'NM_001256252.1', 'Endogenous', 'Hs', False,    42)],
      dtype=[('Name', '<U6'), ('Accession', '<U14'), ('Class', '<U10'), ('Species', '<U2'), ('Annotation', '?'), ('CF330', '<i8')])

数据框形式：

In [26]: pd.DataFrame(_23)                                                                           
Out[26]: 
     Name       Accession       Class Species  Annotation  CF330
0                                                   False     -1
1     A2M     NM_000014.4  Endogenous      Hs       False  11495
2  ACVR1C     NM_145259.2  Endogenous      Hs       False     28
3  ADAM12     NM_003474.5  Endogenous      Hs       False   1020
4  ADGRE1  NM_001256252.1  Endogenous      Hs       False     42

loadtxt 和 genfromtxt 的默认 dtype 是 float。如果文件包含无法转换的字符串，则会在 loadtxt 中出现错误；和nan 在genfromtxt。这些函数的文档很长，但如果你想正确使用它们，值得一读。

np.loadtxt(
    fname,
    dtype=<class 'float'>,      # DEFAULT DTYPE
    comments='#',
    delimiter=None,
    converters=None,
    skiprows=0,
    usecols=None,
    unpack=False,
    ndmin=0,
    encoding='bytes',
    max_rows=None,
)

loadtxt 的其他用法：

In [31]: np.loadtxt(txt.splitlines(), skiprows=1, dtype=str, encoding=None,delimiter=',')            
Out[31]: 
array([['', '', '', '', '', ''],
       ['A2M', 'NM_000014.4', 'Endogenous', 'Hs', '', '11495'],
       ['ACVR1C', 'NM_145259.2', 'Endogenous', 'Hs', '', '28'],
       ['ADAM12', 'NM_003474.5', 'Endogenous', 'Hs', '', '1020'],
       ['ADGRE1', 'NM_001256252.1', 'Endogenous', 'Hs', '', '42']],
      dtype='<U14')
In [32]: np.loadtxt(txt.splitlines(), skiprows=1, dtype=object, encoding=None,delimiter=',')         
Out[32]: 
array([['', '', '', '', '', ''],
       ['A2M', 'NM_000014.4', 'Endogenous', 'Hs', '', '11495'],
       ['ACVR1C', 'NM_145259.2', 'Endogenous', 'Hs', '', '28'],
       ['ADAM12', 'NM_003474.5', 'Endogenous', 'Hs', '', '1020'],
       ['ADGRE1', 'NM_001256252.1', 'Endogenous', 'Hs', '', '42']],
      dtype=object)

Answer 2

使用dtype=object

例如：

FH = np.loadtxt('datafile1.csv', delimiter=',', skiprows=1, dtype=object)
print(FH)