代码花费的时间是python数据框的一个问题

Question

我需要有关以下代码所花费的数据帧相关时间的帮助。 完成大约 2000 条记录的数据集大约需要 20 秒。

def findRe(leaddatadf, keyAttributes, datadf):
    for combs in itertools.combinations(atrList,
        len(atrList)-1):

        v_by =(set(atrList) - set(combs)) # varrying


    grpdatapf=datadf.groupby(combs)
    for name, group in grpdatapf:

        if(group.shape[0]>1):

            tmpgdf = leaddatadf[leaddatadf['unique_id'].astype(float).\
                isin(group['unique_id'].astype(float))]
            if(tmpgdf.shape[0]>1):

                tmpgdf['mprice']=tmpgdf['mprice'].astype(float)
                tmpgdf=tmpgdf.sort('mprice')

                tmpgdf['id'] = tmpgdf['id']
                tmpgdf['desc'] = tmpgdf['description']
                tmpgdf['related_id'] = tmpgdf['id'].shift(-1)
                tmpgdf['related_desc'] = tmpgdf['description'].shift(-1)
                tmpgdf['related_mprice'] = tmpgdf['mprice'].shift(-1)

                tmpgdf['pld'] = np.where(
                    (tmpgdf['related_price'].astype(float) > \
                        tmpgdf['mprice'].astype(float)),
                    (tmpgdf['related_price'].astype(float) - \
                        tmpgdf['mprice'].astype(float)) ,
                    (tmpgdf['mprice'].astype(float) - \
                        tmpgdf['related_mprice'].astype(float)))
                tmpgdf['pltxt'] = np.where(
                    tmpgdf['related_mprice'].astype(float) - \
                        tmpgdf['mprice'].astype(float)>0.0,'<',
                    np.where(tmpgdf['related_mprice'].astype(float)\
                        - tmpgdf['mprice'].astype(float)<0,'>','='))
                tmpgdf['prc_rlt_dif_nbr_p'] = abs(
                    (tmpgdf['pld'].astype(float) / \
                        ((tmpgdf['mprice'].astype(float)))) )
                tmpgdf['keyatr'] = str(atrList)
                tmpgdf['varying'] = np.where(1==1,
                    "".join(v_by ),'')# varrying

                temp = tmpgdf[['id',
            'desc', 'related_id',
            'related_desc', 'pltxt', 'pld',
            'prc_rlt_dif_nbr_p', 'mprice', 'related_mprice',
            'keyatr', 'varying']]

                temp = temp[temp['related_mprice'].astype(float)>=0.0]
                reldf.extend(list(temp.T.to_dict().values()))
    return pd.DataFrame(
                reldf, columns = ['id',
                    'desc', 'related_id',
                    'related_desc', 'pltxt', 'pld',
                    'prc_rlt_dif_nbr_p', 'mprice', 'related_mprice',
                    'keyatr', 'varying'])

Answer 1

请在每行之后打印需要多少毫秒

使用这个https://stackoverflow.com/a/1557584/2655092

并返回花费最多时间的行

Answer 2

您经常使用astype(float)。每次您使用它时，都会创建该系列的副本。当您尝试加载数据框时，您可以尝试在最开始时设置 dtype=float - 这样您只需将系列转换为浮动一次 - 而不是在每次迭代时 :)

如果这有帮助，请告诉我