加快 Pandas 中的多循环数据计算

Question

这是我的问题。以下面的数据框为例：

• 数据框df 有8 列，每一列都有有限值。
• 我要做什么：
  • 一个。按 rows 循环遍历数据框
  • 乙。在每一行中，列B1、B2、B3、B4、B5的值em>, B6 将改为 B* x A

这样的代码：

 for i in range(0,len(df),1):
     col_B = ["B1","B2","B3","B4","B5","B6",]
     for j in range(len(col_B)):
         df.[col_B[j]].iloc[i] = df.[col_B[j]].iloc[i]*df.A.iloc[i]  

在包含 224 行和 9 列的真实数据中，循环遍历所有这些单元格花费了我 0:01:03。

如何提高 Pandas 的循环速度？

任何建议将不胜感激。

Answer 1

你可以先filterDataFrame再多mul：

print(df.filter(like='B').mul(df.A, axis=0))

示例：

import pandas as pd
import numpy as np

df = pd.DataFrame({'A':[1,2,3],
                   'B1':[4,5,6],
                   'B2':[7,8,9],
                   'B3':[1,3,5],
                   'B4':[5,3,6],
                   'B5':[7,4,3],
                   'B6':[1,3,7]})

print (df)
   A  B1  B2  B3  B4  B5  B6
0  1   4   7   1   5   7   1
1  2   5   8   3   3   4   3
2  3   6   9   5   6   3   7

print(df.filter(like='B').mul(df.A, axis=0))
   B1  B2  B3  B4  B5  B6
0   4   7   1   5   7   1
1  10  16   6   6   8   6
2  18  27  15  18   9  21

如果需要列A，请使用concat：

print (pd.concat([df.A, df.filter(like='B').mul(df.A, axis=0)], axis=1))
   A  B1  B2  B3  B4  B5  B6
0  1   4   7   1   5   7   1
1  2  10  16   6   6   8   6
2  3  18  27  15  18   9  21

时间安排：

len(df)=3:

In [416]: %timeit (pd.concat([df.A, df.filter(like='B').mul(df.A, axis=0)], axis=1))
1000 loops, best of 3: 1.01 ms per loop

In [417]: %timeit loop(df)
100 loops, best of 3: 3.28 ms per loop

len(df)=30k:

In [420]: %timeit (pd.concat([df.A, df.filter(like='B').mul(df.A, axis=0)], axis=1))
The slowest run took 4.00 times longer than the fastest. This could mean that an intermediate result is being cached.
100 loops, best of 3: 3 ms per loop

In [421]: %timeit loop(df)
1 loop, best of 3: 35.6 s per loop

计时码：

import pandas as pd

df = pd.DataFrame({'A':[1,2,3],
                   'B1':[4,5,6],
                   'B2':[7,8,9],
                   'B3':[1,3,5],
                   'B4':[5,3,6],
                   'B5':[7,4,3],
                   'B6':[1,3,7]})

print (df)

df = pd.concat([df]*10000).reset_index(drop=True)

print (pd.concat([df.A, df.filter(like='B').mul(df.A, axis=0)], axis=1))

def loop(df):
    for i in range(0,len(df),1):
         col_B = ["B1","B2","B3","B4","B5","B6",]
         for j in range(len(col_B)):
             df[col_B[j]].iloc[i] = df[col_B[j]].iloc[i]*df.A.iloc[i]  
    return df

print (loop(df))