Python BeautifulSoup 在解析 URL 时过滤数据

Question

我每天都在尝试解析这些，在市场开盘之前，我成功获得了列表，但现在我想从基础 url 数据 @987654321 中为“强力买入”和“成交量”> 5000000 添加额外的过滤器@

完整代码如下

import requests
from bs4 import BeautifulSoup

url = "https://www.tradingview.com/markets/stocks-usa/market-movers-gainers/"
siteinfo = requests.get(url)

i = 0
content = siteinfo.content
html = content
parsed_html = BeautifulSoup(html, features="lxml")

doneList = []
for link in parsed_html.find_all('a'):
    a = link.get('href')
    if "symbol" in str(a) and "-" in str(a):
        if i < 25:
            i += 1
        else:
            x = a.split("-")
            x = x[1].split("/")
            doneList.append(x[0])
            i += 1

print(doneList)

Answer 1

在这种特殊情况下，您最好使用带有多个条件和过滤器的 pandas：

import pandas as pd
url = 'https://www.tradingview.com/markets/stocks-usa/market-movers-gainers/'
df = pd.read_html(url)[0]

#create a helper function as a filter - it returns a series of boolean values
def filter_out(row):
    #Unnamed: 4 is the buy recommendation and the next one is volume
    if 'Strong' in row['Unnamed: 4'] and 'M' in row['Unnamed: 5']:
        #since you're using a 5M volume as condition, you have to check for its existence:
        if (int(row['Unnamed: 5'].split('.')[0])>5):
            return True
        else:
            return False
    else:
        return False
#use the boolean values to filter the dataframe:
bulls = df.apply(filter_out, axis=1)
df[bulls]

输出（请原谅格式）：

     Unnamed: 0            Unnamed: 1   Unnamed: 2  Unnamed: 3  Unnamed: 4  Unnamed: 5  Unnamed: 6  Unnamed: 7  Unnamed: 8  Unnamed: 9  Unnamed: 10
0   M MRIN Marin Software Incorporated  7.50    96.85%  3.69    Strong Buy  263.387M    41.786M     —   -1.59   162.00  Technology Services
2   NTLA Intellia Therapeutics, Inc.    133.43  50.21%  44.60   Strong Buy  21.740M     6.054B  —   -2.46   312.00  Health Technology
3   A AUUD Auddia Inc.  5.89    43.66%  1.79    Strong Buy  36.281M     46.296M     —   —   11.00   Technology Services

等等。然后，您可以更改列名称或进行其他处理。

编辑：

要仅获取这些公司的股票代码，请使用：

ticks = df[bulls]['Unnamed: 0'].to_list()
for tick in ticks:
    print(tick.split('  ')[-2])

输出：

MRIN
NTLA
AUUD
WTT

等等

Answer 2

首先，退出代码 0 表示您的代码中没有错误。因此，索引超出范围错误必须来自打印语句print(e)（已处理异常）。

查看您的代码，这是列表索引超出范围错误最容易受到攻击的部分。

quote = get_quote(stock)

我不知道get_quote的内部机制，但我猜这就是错误发生的地方。