【问题标题】:Python dealing with 2 for loops togetherPython一起处理2个for循环
【发布时间】:2016-01-19 16:37:26
【问题描述】:

好的。下面是完整的代码。我想循环两个不同的数据集,每个数据集一年。获取每个冰雹概率值的 hailindx 值百分位数并绘制它们。由于我需要循环这两个 1 年的数据集,但速度非常慢。

from matplotlib import pyplot as plt
from matplotlib import mlab
import netCDF4 as net
import numpy as np
import itertools    days=["01","02","03","04","05","06","07","08","09","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31"]
months=["01","02","03","04","05","06","07","08","09","10","11","12"]
hp_values=range(0,100)
for value in hp_values:
    value1=[]
    print value
    for month,day in itertools.product(months,days):
        print month,day
        try:
            hailindx1="/Trunk/2015HailIndx/HailIndx2015%s%sL0S_CONUS.nc"%(month,day)
            hailprob1="/Trunk/2015/aerHailProb2015%s%s.nc" %(month,day)
            hailindx=net.Dataset(hailindx1)
            hailprob=net.Dataset(hailprob1)
            hp=hailprob.variables['HailProb'][:]
            hs=hailindx.variables['HailIndx'][:]
            p=[0.05,0.1,0.2]
            hp=np.array(hp)
            hs=np.array(hs)
            mask=(hp>0) & (hs>0)
            hs=hs[mask]
            hp=hp[mask]
            value2=hs[hp==value]
            if len(value2)>0:
                value1.append(value2)
            else:
                continue 
        except:
            continue
    value_list=[value,value,value]
    print value_list
    if len(value1)>0:
        perc=np.percentile(value1,p)
        plt.plot(value_list,perc,marker='o',color='r')
    else:
        continue

plt.xlabel('HailProb')
plt.ylabel('HailIndx')
plt.show()

如果有人知道如何使循环更快。

【问题讨论】:

  • 速度慢是因为循环还是因为你在循环内做了什么?
  • itertools.product 不会提供加速...仅供参考
  • 你是对的,刚刚试了,还是一样的..... :(
  • 它很慢,因为你对月/日所做的事情......迭代这几乎是即时的
  • 你想用几个月和几天的时间来说它太慢了?

标签: python loops itertools


【解决方案1】:

您可以使用itertools.product 获取所有组合。像这样:

for month, day in itertools.product(months, days):
    ...do something...

【讨论】:

  • 这不会给他一个加速......更糟糕的是它更不清晰和清楚他在做什么(即使这是 OP 要求的:P)
【解决方案2】:

您可以使用itertools 中的product() 函数:

from itertools import product

months=["01","02","03","04","05","06","07","08","09","10","11","12"]
days=["01","02","03","04","05","06","07","08","09","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31"]

answer = list(product(months, days))

输出

[('01', '01'),
 ('01', '02'),
 ('01', '03'),
 ('01', '04'),
 ('01', '05'),
 ...
 ('12', '28'),
 ('12', '29'),
 ('12', '30'),
 ('12', '31')]

然后您可以根据需要迭代 answer 变量。

【讨论】:

    【解决方案3】:

    请注意,您的循环将返回不可能的日期,例如 2015/02/31。直接使用日期可能会更好。

    还请注意,您正在加载和过滤每个数据文件 100 次;你真的只需要加载一次,如果你聪明的话,你可能可以一次过滤它。

    你的hp_values 也应该是range(0, 101) 即100 是一个可能的值?

    类似

    from datetime import date, timedelta
    import numpy as np
    
    YEAR = 2015
    # using datetime.strftime format codes
    INDEX_FILE = "/Trunk/%YHailIndx/HailIndx%Y%m%dL0S_CONUS.nc"
    PROB_FILE  = "/Trunk/%Y/aerHailProb%Y%m%d.nc"
    
    def date_range(start_date, end_date, step=timedelta(1)):
        day = start_date
        while day < end_date:
            yield day
            day += step
    
    def main():
        start = date(YEAR,     1, 1)
        end   = date(YEAR + 1, 1, 1)
        for day in date_range(start, end):
            # load index file
            try:
                index_file = day.strftime(INDEX_FILE)
                index_data = net.Dataset(index_file)
            except RuntimeError as re:
                print(re)
                print("Failed to load index file:", index_file)
                continue                
            # load probability file
            try:
                prob_file = day.strftime(PROB_FILE)
                prob_data = net.Dataset(prob_file)
            except RuntimeError as re:
                print(re)
                print("Failed to load probability file:", prob_file)
                continue                
            # start calculating
            index = np.array(index_data.variables['HailIndx'])
            prob  = np.array(prob_data .variables['HailProb'])
            #
            # Here I started to get a bit lost trying to follow what
            # you are doing; a sample index file and probability file
            # would probably help in debugging, as would a better
            # description of exactly what you are trying to do to
            # the numbers ;-)
            #
    
    if __name__ == "__main__":
        main()
    

    【讨论】:

      猜你喜欢
      • 2013-12-10
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2020-09-25
      • 2022-10-06
      • 2018-08-06
      • 2016-03-31
      相关资源
      最近更新 更多