如何在 python 中从大型 CSV 文件中读取所需的行

Question

我正在尝试从 CSV 文件中搜索数据并将数据传递给另一个 python 代码。 CSV 文件有 100000 多行，我想根据我的选择传递请求的数据。

实际代码：

input_file = 'trusted.csv'
users = []
with open(input_file, encoding='UTF-8') as f:
    rows = csv.reader(f,delimiter=",",lineterminator="\n")
    next(rows, None)
    for row in rows:
      user = {}
      user['username'] = row[0]
      user['id'] = int(row[1])
      user['access_hash'] = int(row[2])
      user['name'] = row[3]
      users.append(user)

将数据解析为代码：

g_index = input("Enter a Number: ")
target_group=groups[int(g_index)]
target_group.access_hash

实际代码将解析 CSV 文件中的 所有 行，我正在尝试为可以传递数据的 python 代码找到一个解决方案 - 例如从 11 到 20 行，从 50 到 100行也一样。

我尝试了以下代码，但在将数据解析为另一个 python 代码时收到错误：

import CSV


input_file = 'lucky280.csv'

start = 10
stop = start + 10
users = []
with open(input_file, encoding='UTF-8') as f:
    rows = csv.reader(f,delimiter=",",lineterminator="\n")
    for  i, line in enumerate(rows):
        if i >= start:
            users.append(line)
        if i > stop:
            break
            for row in rows:
              user = {}
              user['username'] = row[0]
              user['id'] = int(row[1])
              user['access_hash'] = int(row[2])
              user['name'] = row[3]
              users.append(user)

错误： 回溯（最近一次通话最后）： 文件“”，第 10 行，在 print ("添加 {}".format(user['id'])) TypeError: 列表索引必须是整数或切片，而不是 str

如果我使用实际代码，文件读取将正常工作，但它会解析文件中的所有数据。

请帮忙！

推荐后我也试过了

input_file = 'lucky280.csv'
users = []
from itertools import islice
with open(input_file, encoding='UTF-8') as f:
    rows = csv.reader(f,delimiter=",",lineterminator="\n")
    rowiter = islice(rows, 3, 5)
    for item in rowiter:
            for row in rows:
              user = {}
              user['username'] = row[0]
              user['id'] = int(row[1])
              user['access_hash'] = int(row[2])
              user['name'] = row[3]
              users.append(user)

出现以下错误

IndexError Traceback (most recent call last)

<ipython-input-108-9f4099c2e53d> in <module>()
     10               user = {}
     11               user['username'] = row[0]
---> 12               user['id'] = int(row[1])
     13               user['access_hash'] = int(row[2])
     14               user['name'] = row[3]

IndexError: list index out of range

Answer 1

你可以从itertools使用islice

所以这里我有一个示例 csv 文件

    X      Y
0  21  test3
1   8  test1
2  75  test1
3  26  test2
4  98  test3
5  63  test3
6  65  test3
7  39  test3
8  74  test1
9  26  test2

假设我只想要第 3 行和第 4 行

>>> from itertools import islice
>>> with open('test.csv') as f:
...     rows = csv.reader(f)
...     rowiter = islice(rows, 3, 5)
...     for item in rowiter:
...             print(item)

给我以下输出

['2', '75', 'test1']
['3', '26', 'test2']

更新

input_file = 'trusted.csv'
start = 10
stop = start + 10
users = []

with open(input_file, encoding='UTF-8') as f:
    rows = csv.reader(f,delimiter=",",lineterminator="\n")    
    rowiter = islice(rows, start, stop)
    for row in rowiter :
      user = {}
      user['username'] = row[0]
      user['id'] = int(row[1])
      user['access_hash'] = int(row[2])
      user['name'] = row[3]
      users.append(user)