缺少写文件的循环答案

【问题标题】：Missing a loop to write a file缺少写文件的循环
【发布时间】：2016-06-28 18:04:55
【问题描述】：

import urllib.request
import re
import csv
import pandas as pd
from bs4 import BeautifulSoup

stocklist = ['aapl','goog','fb','amzn','COP']
for stocklist in stocklist:
    optionsUrl = urllib.request.urlopen('http://finance.yahoo.com/q?s='+stocklist).read()
    soup = BeautifulSoup(optionsUrl)
    stocksymbol = ['Symbol:',''+stocklist+'']
    optionsTable = [stocksymbol]+[
        [x.text for x in y.parent.contents]
        for y in soup.findAll('td', attrs={'class': 'yfnc_tabledata1','rtq_table': ''})
    ]
    print(optionsTable)
    my_df = pd.DataFrame(optionsTable).T
    my_df.to_csv('test.csv', index=False, header=False)

我有这段代码。有人建议我使用熊猫。我能够将列表中的数据写入 CSV 文件。但是 CSV 文件只有 COP 的数据，但没有其他股票的数据（csv 文件只有一行数据，我假设它被覆盖了）。有人可以告诉我我缺少什么或修复此代码吗？ print(optionsTable) 虽然打印了 4 行..

这是输出：

[['Symbol:', 'aapl'], ['Prev Close:', '99.65'], ['Open:', '98.51'], ['Bid:', '98.95 x 1700'], ['Ask:', '98.96 x 1200'], ['1y Target Est:', '124.90'], ['Beta:', '1.48679'], ['Earnings Date:', 'Jul 19 - Jul 25 (Est.)'], ["Day's Range:", '98.48 - 99.35'], ['52wk Range:', '89.47 - 132.97'], ['Volume:', '28,454,663'], ['Avg Vol (3m):', '38,261,900'], ['Market Cap:', '541.57B'], ['P/E (ttm):', '11.01'], ['EPS (ttm):', '8.98'], ['Div & Yield:', '2.28 (2.30%) '], ['Forward P/E (1 yr):', '10.86'], ['P/S (ttm):', '2.40'], ['Ex-Dividend Date:', '05-May-16'], ['Annual EPS Est\n                      (Sep-16)\n                    :', '8.28'], ['Quarterly EPS Est\n                      (Jun-16)\n                    :', '1.39'], ['Mean Recommendation*:', '1.8'], ['PEG Ratio (5 yr expected):', '1.30']]
[['Symbol:', 'goog'], ['Prev Close:', '728.58'], ['Open:', '719.47'], ['Bid:', '717.60 x 400'], ['Ask:', '717.96 x 100'], ['1y Target Est:', '924.83'], ['Beta:', '1.032'], ['Next Earnings Date:', 'N/A'], ["Day's Range:", '716.43 - 725.86'], ['52wk Range:', '515.18 - 789.87'], ['Volume:', '1,050,710'], ['Avg Vol (3m):', '1,781,050'], ['Market Cap:', '493.43B'], ['P/E (ttm):', '29.25'], ['EPS (ttm):', '24.58'], ['Div & Yield:', 'N/A (N/A) '], ['Forward P/E (1 yr):', 'N/A'], ['P/S (ttm):', '6.41'], ['Ex-Dividend Date:', 'N/A'], ['Annual EPS Est\n                      (Dec-16)\n                    :', 'N/A'], ['Quarterly EPS Est\n                      (Jun-16)\n                    :', 'N/A'], ['Mean Recommendation*:', '1.8'], ['PEG Ratio (5 yr expected):', 'N/A']]
[['Symbol:', 'fb'], ['Prev Close:', '118.56'], ['Open:', '117.52'], ['Bid:', '116.39 x 800'], ['Ask:', '116.40 x 500'], ['1y Target Est:', '142.87'], ['Beta:', '0.840485'], ['Earnings Date:', 'Jul 27 - Aug 1 (Est.)'], ["Day's Range:", '116.26 - 118.11'], ['52wk Range:', '72.00 - 121.08'], ['Volume:', '17,257,639'], ['Avg Vol (3m):', '25,746,700'], ['Market Cap:', '333.25B'], ['P/E (ttm):', '71.26'], ['EPS (ttm):', '1.64'], ['Div & Yield:', 'N/A (N/A) '], ['Forward P/E (1 yr):', '25.25'], ['P/S (ttm):', '17.16'], ['Ex-Dividend Date:', 'N/A'], ['Annual EPS Est\n                      (Dec-16)\n                    :', 'N/A'], ['Quarterly EPS Est\n                      (Jun-16)\n                    :', 'N/A'], ['Mean Recommendation*:', '1.7'], ['PEG Ratio (5 yr expected):', 'N/A']]
[['Symbol:', 'amzn'], ['Prev Close:', '727.65'], ['Open:', '722.35'], ['Bid:', '716.25 x 500'], ['Ask:', '716.50 x 100'], ['1y Target Est:', '800.92'], ['Beta:', '1.6465'], ['Earnings Date:', 'Jul 21 - Jul 25 (Est.)'], ["Day's Range:", '714.21 - 724.98'], ['52wk Range:', '422.64 - 731.50'], ['Volume:', '3,161,899'], ['Avg Vol (3m):', '3,948,360'], ['Market Cap:', '338.47B'], ['P/E (ttm):', '295.70'], ['EPS (ttm):', '2.43'], ['Div & Yield:', 'N/A (N/A) '], ['Forward P/E (1 yr):', '72.29'], ['P/S (ttm):', '3.03'], ['Ex-Dividend Date:', 'N/A'], ['Annual EPS Est\n                      (Dec-16)\n                    :', '5.38'], ['Quarterly EPS Est\n                      (Jun-16)\n                    :', '1.10'], ['Mean Recommendation*:', '1.8'], ['PEG Ratio (5 yr expected):', '2.43']]
[['Symbol:', 'COP'], ['Prev Close:', '46.57'], ['Open:', '45.90'], ['Bid:', '44.47 x 1300'], ['Ask:', '44.48 x 2300'], ['1y Target Est:', '51.23'], ['Beta:', '1.42252'], ['Earnings Date:', 'Jul 28 - Aug 1 (Est.)'], ["Day's Range:", '44.26 - 46.12'], ['52wk Range:', '31.05 - 64.13'], ['Volume:', '8,217,057'], ['Avg Vol (3m):', '8,947,330'], ['Market Cap:', '55.11B'], ['P/E (ttm):', 'N/A'], ['EPS (ttm):', '-4.98'], ['Div & Yield:', '1.98 (4.16%) '], ['Forward P/E (1 yr):', '143.48'], ['P/S (ttm):', '2.11'], ['Ex-Dividend Date:', '18-May-16'], ['Annual EPS Est\n                      (Dec-16)\n                    :', '-2.26'], ['Quarterly EPS Est\n                      (Jun-16)\n                    :', '-0.67'], ['Mean Recommendation*:', '2.5'], ['PEG Ratio (5 yr expected):', '0.37']]

【问题讨论】：

这段代码非常混乱。使用for stocklist in stocklist: 将stocklist 的含义从股票列表更改为该列表中的单个项目（它不会破坏迭代，因为在stocklist 反弹之前已经开始了，但它仍然非常糟糕想法）。
我正在尝试从雅虎财经下载股票数据列表
该脚本主要下载市值、股息、每股收益等股票信息。如果我们运行该脚本，输出将显示列名和数据。
您可能想尝试使用pandas_datareader，它将为您完成所有工作

标签： python pandas

【解决方案1】：

每次循环时都会覆盖 csv。您应该收集所有数据并在循环后将它们写入 csv：

stocklist = ['aapl','goog','fb','amzn','COP']
columns = []
data = []
for s in stocklist:
    optionsUrl = urllib.request.urlopen('http://finance.yahoo.com/q?s='+s).read()
    soup = BeautifulSoup(optionsUrl, "html.parser")
    stocksymbol = ['Symbol:', s]
    optionsTable = [stocksymbol]+[
    [x.text for x in y.parent.contents]
    for y in soup.findAll('td', attrs={'class': 'yfnc_tabledata1','rtq_table': ''})
    ]

    if not columns:
        columns = [o[0] for o in optionsTable]
    data.append(o[1] for o in optionsTable)

# create DataFrame from data
df = pd.DataFrame(data, columns=columns)
df.to_csv('test.csv', index=False)

【讨论】：

当你不进行繁重的数据操作时，你可以考虑使用 python 的csv 模块

【解决方案2】：

您可以通过执行附加到文件

with open('test.csv', 'a') as f:
    my_df.to_csv(f, header=False)

这个

【讨论】：

【解决方案3】：

我建议您使用pandas-datareader，它专为您将要做的事情而设计。

这是一个小演示：

from datetime import datetime
import pandas_datareader.data as wb

stocklist = ['AAPL','GOOG','FB','AMZN','COP']

start = datetime(2016,6,8)
end = datetime(2016,6,11)

p = wb.DataReader(stocklist, 'yahoo',start,end)

p - 是熊猫panel，我们可以用它做有趣的事情：

让我们看看我们的面板中有什么

In [388]: p.axes
Out[388]:
[Index(['Open', 'High', 'Low', 'Close', 'Volume', 'Adj Close'], dtype='object'),
 DatetimeIndex(['2016-06-08', '2016-06-09', '2016-06-10'], dtype='datetime64[ns]', name='Date', freq='D'),
 Index(['AAPL', 'AMZN', 'COP', 'FB', 'GOOG'], dtype='object')]

In [389]: p.keys()
Out[389]: Index(['Open', 'High', 'Low', 'Close', 'Volume', 'Adj Close'], dtype='object')

选择数据

In [390]: p['Adj Close']
Out[390]:
                 AAPL        AMZN        COP          FB        GOOG
Date
2016-06-08  98.940002  726.640015  47.490002  118.389999  728.280029
2016-06-09  99.650002  727.650024  46.570000  118.559998  728.580017
2016-06-10  98.830002  717.909973  44.509998  116.620003  719.409973

In [391]: p['Volume']
Out[391]:
                  AAPL       AMZN        COP          FB       GOOG
Date
2016-06-08  20812700.0  2200100.0  9596700.0  14368700.0  1582100.0
2016-06-09  26419600.0  2163100.0  5389300.0  13823400.0   985900.0
2016-06-10  31462100.0  3409500.0  8941200.0  18412700.0  1206000.0

In [394]: p[:,:,'AAPL']
Out[394]:
                 Open       High        Low      Close      Volume  Adj Close
Date
2016-06-08  99.019997  99.559998  98.680000  98.940002  20812700.0  98.940002
2016-06-09  98.500000  99.989998  98.459999  99.650002  26419600.0  99.650002
2016-06-10  98.529999  99.349998  98.480003  98.830002  31462100.0  98.830002

In [395]: p[:,'2016-06-10']
Out[395]:
            Open        High         Low       Close      Volume   Adj Close
AAPL   98.529999   99.349998   98.480003   98.830002  31462100.0   98.830002
AMZN  722.349976  724.979980  714.210022  717.909973   3409500.0  717.909973
COP    45.900002   46.119999   44.259998   44.509998   8941200.0   44.509998
FB    117.540001  118.110001  116.260002  116.620003  18412700.0  116.620003
GOOG  719.469971  725.890015  716.429993  719.409973   1206000.0  719.409973

【讨论】：