熊猫数据框 - RemoteDataError - Python

Question

我正在尝试从雅虎财经中提取数据。

这是我得到的错误：

File "banana.py", line 35, in <module>
    data = web.DataReader(ticker, "yahoo", datetime(2011,1,1), datetime(2015,12,31))
  File "C:\Users\ll\Anaconda2\lib\site-packages\pandas_datareader\data.py", line 94, in DataReader
    session=session).read()
  File "C:\Users\ll\Anaconda2\lib\site-packages\pandas_datareader\yahoo\daily.py", line 77, in read
    df = super(YahooDailyReader, self).read()
  File "C:\Users\ll\Anaconda2\lib\site-packages\pandas_datareader\base.py", line 173, in read
    df = self._read_one_data(self.url, params=self._get_params(self.symbols))
  File "C:\Users\ll\Anaconda2\lib\site-packages\pandas_datareader\base.py", line 80, in _read_one_data
    out = self._read_url_as_StringIO(url, params=params)
  File "C:\Users\ll\Anaconda2\lib\site-packages\pandas_datareader\base.py", line 91, in _read_url_as_StringIO
    response = self._get_response(url, params=params)
  File "C:\Users\ll\Anaconda2\lib\site-packages\pandas_datareader\base.py", line 117, in _get_response
    raise RemoteDataError('Unable to read URL: {0}'.format(url))
pandas_datareader._utils.RemoteDataError: Unable to read URL: http://ichart.finance.yahoo.com/table.csv

当我从 .csv 文件而不是代码列表中读取时出现错误：

这行得通：

for ticker in ['MSFT']:

这不是：

input_file = open("testlist.csv", 'r')
for ticker in input_file: 

我什至添加了异常（见下文）但仍然无法正常工作：

    except RemoteDataError:
        print("No information for ticker '%s'" % t)
        continue

    except IndexError:
        print("Something went wacko for ticker '%s', trying again..." % t)
        continue

    except Exception, e:
        continue

    except:
        print "Can't find ", ticker

我的代码：

from datetime import datetime
from pandas_datareader import data, wb
import pandas_datareader.data as web
import pandas as pd
from pandas_datareader._utils import RemoteDataError
import csv
import sys 
import os

class MonthlyChange(object):
    months = { 0:'JAN', 1:'FEB', 2:'MAR', 3:'APR', 4:'MAY',5:'JUN', 6:'JUL', 7:'AUG', 8:'SEP', 9:'OCT',10:'NOV', 11:'DEC' }

def __init__(self,month):
    self.month = MonthlyChange.months[month-1]
    self.sum_of_pos_changes=0
    self.sum_of_neg_changes=0
    self.total_neg=0
    self.total_pos=0
def add_change(self,change):
    if change < 0:
        self.sum_of_neg_changes+=change
        self.total_neg+=1
    elif change > 0:
        self.sum_of_pos_changes+=change
        self.total_pos+=1
def get_data(self):
    if self.total_pos == 0:
        return (self.month,0.0,0,self.sum_of_neg_changes/self.total_neg,self.total_neg)
    elif self.total_neg == 0:
        return (self.month,self.sum_of_pos_changes/self.total_pos,self.total_pos,0.0,0)
    else:
        return (self.month,self.sum_of_pos_changes/self.total_pos,self.total_pos,self.sum_of_neg_changes/self.total_neg,self.total_neg)

input_file = open("Companylistnysenasdaq.csv", 'r')

for ticker in input_file:  #for ticker in input_file:


print(ticker)
data = web.DataReader(ticker, "yahoo", datetime(2011,1,1), datetime(2015,12,31))
data['ymd'] = data.index
year_month = data.index.to_period('M')
data['year_month'] = year_month
first_day_of_months = data.groupby(["year_month"])["ymd"].min()
first_day_of_months = first_day_of_months.to_frame().reset_index(level=0)
last_day_of_months = data.groupby(["year_month"])["ymd"].max()
last_day_of_months = last_day_of_months.to_frame().reset_index(level=0)
fday_open = data.merge(first_day_of_months,on=['ymd'])
fday_open = fday_open[['year_month_x','Open']]
lday_open = data.merge(last_day_of_months,on=['ymd'])
lday_open = lday_open[['year_month_x','Open']]

fday_lday = fday_open.merge(lday_open,on=['year_month_x'])
monthly_changes = {i:MonthlyChange(i) for i in range(1,13)}
for index,ym, openf,openl in fday_lday.itertuples():
    month = ym.strftime('%m')
    month = int(month)
    diff = (openf-openl)/openf
    monthly_changes[month].add_change(diff)
changes_df = pd.DataFrame([monthly_changes[i].get_data() for i in monthly_changes],columns=["Month","Avg Inc.","Inc","Avg.Dec","Dec"])




t = ticker.strip()                                       
j = 0
while j < 13:

    try:
        if len(changes_df.loc[changes_df.Inc > 2,'Month']) != 0:
            print ticker
            print ("Increase Months: ")
            print (changes_df.loc[changes_df.Inc > 2,'Month'])

        if len(changes_df.loc[changes_df.Dec > 2,'Month']) != 0:
            print ticker
            print ("Decrease Months: ")
            print (changes_df.loc[changes_df.Dec > 2,'Month'])

        j += 13


    except RemoteDataError:
        print("No information for ticker '%s'" % t)
        j += 13
        continue

    except IndexError:
        print("Something went googoo for ticker '%s', trying again..." % t)
        j += 1
        time.sleep(30)
        continue

    except Exception, e:
        j+=13
        time.sleep(30)
        continue

    except:
        print "Can't find ", ticker


input_file.close()

Answer 1

pandas_datareader 会在 yahoo 未通过其 API 提供相关ticker 的数据时引发此错误。

在读取您的 .csv 文件时，您包含换行符，因此 pandas_datareader 无法识别代码。

data = web.DataReader(ticker.strip('\n'), "yahoo", datetime(2011, 1, 1), datetime(2015, 12, 31))

当我创建一个在第一列中列出代码的文件时有效。

可能更容易做到：

tickers = pd.read_csv('Companylistnysenasdaq.csv')
for ticker in tickers.iloc[:, 0].tolist():

假设您的文件是一个简单的列表，第一列中有代码。可能需要read_csv 中的header=None，具体取决于您的文件格式。

要处理错误，您可以：

from pandas_datareader._utils import RemoteDataError

try:
    stockData = DataReader(ticker, 'yahoo', datetime(2015, 1, 1), datetime.today())
except RemoteDataError:
    # handle error

Answer 2

试试这个： 您打开文件但没有.read() 。

 input_file = open("testlist.csv", 'r').read()

请运行这些：

input_file = open("Companylistnysenasdaq.csv", 'r').read()
for x in input_file: print x

input_file = open("Companylistnysenasdaq.csv", 'r')
for x in input_file: print x