带有块大小的熊猫 read_csv

Question

# calculate CTR
count_all = 0
count_4 = 0
for df in pd.read_csv( open("%s/tianchi_fresh_comp_train_user.csv" % 
root_path,'r'), chunksize=10000):
     try:
         count_user = df['behavior_type'].value_counts()
         count_all += count_user[1]+count_user[2]+count_user[3]+count_user[4]
         count_4 += count_user[4]
     except StopIteration:
         print("Iteration is stopped.")

# CTR
print(count_all)
print(count_4)

错误信息

但如果我将 chunksize 从 10000 修改为 100000。 chunksize = 100000, 没关系，没问题

为什么，我设置chunksize = 10000，有错误？

Answer 1

count_all = 0
count_4 = 0
for df in pd.read_csv( open("%s/tianchi_fresh_comp_train_user.csv" % root_path,'r'), 
chunksize=10000):
    try:
        count_user = df['behavior_type'].value_counts()
        for i in range(5):
            if i not in count_user.index: count_user[i] = 0
            else:
               count_all += count_user[i]
        count_4 += count_user[4]
    except StopIteration:
          print("Iteration is stopped.")

我修改了代码，现在可以了，当chunksize=10000时，没问题。