循环将多个列表附加到数据框python中

Question

我必须 requests.get() 来自雅虎的两个 url 使用 YQL 返回一个 json 对象。我正在取回我存储到list() 中的json 对象。然后我循环解析数据并创建一个 dic，然后创建一个 pandas 数据框。碰巧只有一个列表被附加到数据框中。似乎在最后一次迭代中，第二个列表覆盖了第一个列表。在这一点上，我无法弄清楚如何在列表上迭代到 append() 列表的两个元素。这是我的代码...

import requests
import pandas as pd


urls = ['https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.historicaldata%20where%20symbol%20in%20(%22DIA%22%2C%22SPY%22%2C%22IWN%22)%20and%20startDate%20%3D%20%222015-01-01%22%20and%20endDate%20%3D%20%222015-10-31%22&format=json&diagnostics=true&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback=',
        'https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.historicaldata%20where%20symbol%20in%20(%22DIA%22%2C%22SPY%22%2C%22IWN%22)%20and%20startDate%20%3D%20%222015-11-01%22%20and%20endDate%20%3D%20%222016-08-31%22&format=json&diagnostics=true&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback=']


for url in urls:
    data = requests.get(url)
    data_json = data.json()

    quote_list = []
    for quote in data_json['query']['results']['quote']:
        quote_dic = {'symbol': quote['Symbol'],
                 'date': quote['Date'],
                 'volume': quote['Volume'],
                 'low': quote['Low'],
                 'high': quote['High'],
                 'open': quote['Open'],
                 'close': quote['Close'],
                 'adj_close': quote['Adj_Close']}

        quote_list.append(quote_dic)

    quote_df = pd.DataFrame(quote_list)

quote_df.to_csv('stocks.csv')

我需要能够将整个 list() 附加到数据框中。这段代码的修复方法是什么？

Answer 1

只需创建一个数据帧列表，并在循环结束时将它们连接起来：

df_list = []
for url in urls:
    data = requests.get(url)
    data_json = data.json()

    df = pd.DataFrame(data_json['query']['results']['quote'])
    df_list.append(df)

quote_df = pd.concat(df_list)
quote_df.to_csv('stocks.csv')

Answer 2

这个解决方案怎么样？

import urllib
import re
import json

symbolslist = open("C:/Users/your_path_here/Desktop/stock_symbols.txt").read()
symbolslist = symbolslist.split("\n")

for symbol in symbolslist:
    myfile = open("C:/Users/your_path_here/Desktop/" +symbol +".txt", "w+")
    myfile.close()

    htmltext = urllib.urlopen("http://www.bloomberg.com/markets/chart/data/1D/"+ symbol+ ":US")
    data = json.load(htmltext)
    datapoints = data["data_values"]

    myfile = open("C:/Users/rshuell001/Desktop/symbols/" +symbol +".txt", "a")
    for point in datapoints:
        myfile.write(str(symbol+","+str(point[0])+","+str(point[1])+"\n"))
    myfile.close()

在这个文件中“C:/Users/your_path_here/Desktop/symbols/amex.txt” 你有以下代码

ibm
sbux
msft