尝试运行 sql 查询以填充多个数据帧但得到空 dfs

Question

我有一个 sql 查询，需要在不同的时间段分别运行。我想循环运行它并填充所有数据帧。如果我一个接一个地运行查询，它工作正常，但循环输出一大堆空的 dfs。

conn = pyodbc.connect("my connection details")

# time periods
may_jun_19 = [dt.date(day=1, month=5, year=2019), dt.date(day=1, month=7, year=2019)]
jul_19 = [dt.date(day=1, month=7, year=2019), dt.date(day=1, month=8, year=2019)]
aug_nov_19 = [dt.date(day=1, month=8, year=2019), dt.date(day=1, month=12, year=2019)]
dec_19 = [dt.date(day=1, month=12, year=2019), dt.date(day=1, month=1, year=2020)]
jan_feb_20 = [dt.date(day=1, month=1, year=2020), dt.date(day=1, month=3, year=2020)]
mar_may_20 = [dt.date(day=1, month=3, year=2020), dt.date(day=20, month=5, year=2020)]

months = [may_jun_19, jul_19, aug_nov_19, dec_19, jan_feb_20, mar_may_20]

# Initialise empty dataframes and put them in a list
mj19 = j19 = an19 = d19 = jf20 = mm20 = pd.DataFrame()
dfs = [mj19, j19, an19, d19, jf20, mm20]

# Run a query under a for cycle
for df, month in zip(dfs, months):
    df = pd.read_sql_query(query, conn, params=month)

如果我运行这段代码，我会得到一个名为 df 的数据帧，尽管我从未初始化过这个名称，以及一大堆空数据帧。

这是我要运行的查询：

 query = """select source_location,
                  dest_location,
                  sum(best_tonnes)
           from mq2.v_movement_summary 
           where start_time >= ? and 
                  end_time < ? and
                  source_location = '18_TO_JB CRUSH' or
                  source_location like 'AP11%' and
                  mine_site_code = 'JB' and 
                  type = 'Movement'
           group by source_location,
                    dest_location"""

我在这里做错了什么？

Answer 1

A.这不符合您的预期：

# Initialise empty dataframes and put them in a list
mj19 = j19 = an19 = d19 = jf20 = mm20 = pd.DataFrame()
dfs = [mj19, j19, an19, d19, jf20, mm20]

您正在创建 6 个对同一个空数据框的引用，并将它们放在一个您不再使用的列表中。

B.这没什么用，很好；

# Run a query under a for cycle
for df, month in zip(dfs, months):
    df = pd.read_sql_query(query, conn, params=month)

您从查询中创建一个数据框。就是这样。结束。
df 变量在每个“循环”中都会被覆盖，而您只剩下循环的最后一个数据帧。

C.解决方案；

# ... your time periods ...

months = [may_jun_19, jul_19, aug_nov_19, dec_19, jan_feb_20, mar_may_20]

# initialize an empty list
list_ = []

# run your queries
for df, month in zip(dfs, months):
    df = pd.read_sql_query(query, conn, params=month)

    # append the df created by your query to the list
    list_.append(df)

现在您有了一个数据框列表。