使用 BeautifulSoup 的网页文本答案

【问题标题】：Text from webpage using BeautifulSoup使用 BeautifulSoup 的网页文本
【发布时间】：2018-11-20 17:24:10
【问题描述】：

我正在尝试使用 Python 从https://markets.cboe.com/europe/equities/market_share/index/all/ 中提取一些数据

特别是“市场未显示总成交量”的数字，我尝试了多种使用 BeautifulSoup 的方法，但似乎没有一个能让我到达那里。

有什么想法吗？

【问题讨论】：

请分享您到目前为止所做的事情并指出失败的地方。
我在该页面上找不到任何显示“市场未显示总成交量”的内容

标签： python http beautifulsoup

【解决方案1】：

我建议给 pandas html 阅读器一个机会：

import pandas as pd

# Read in all tables at this address as pandas dataframes
results = pd.read_html('https://markets.cboe.com/europe/equities/market_share/index/all')

# Grab the second table founds
df = results[1]
# Set the first column as the index
df = df.set_index(0)
# Switch columns and indexes
df = df.T
# Drop any columns that have no data in them
df = df.dropna(how='all', axis=1)
# Set the column under "Displayed Price Venues" as the index
df = df.set_index('Displayed Price Venues')
# Switch columns and indexes again
df = df.T

# Aesthetic. Don't like having an index name myself! 
del df.index.name

# Separate the three subtables from each other!  
displayed = df.iloc[0:18]
non_displayed = df.iloc[18:-1]
total = df.iloc[-1]

您也可以以更紧凑的方式执行此操作（相同的代码但不分解步骤）：

import pandas as pd

# Read in all tables at this address as pandas dataframes
results = pd.read_html('https://markets.cboe.com/europe/equities/market_share/index/all')

# Do all the stuff above in one go
df = results[1].set_index(0).T.dropna(how='all',axis=1).set_index('Displayed Price Venues').T

# Aesthetic. Don't like having an index name myself! 
del df.index.name

# Separate the three subtables from each other!  
displayed = df.iloc[0:18]
non_displayed = df.iloc[18:-1]
total = df.iloc[-1]

【讨论】：

【解决方案2】：

问题是id 不断变化。否则，我会使用它但不能。假设输出值是您正在寻找的，这应该可以工作，只要内容不改变或移动。

from bs4 import BeautifulSoup as bs
import requests

url = 'https://markets.cboe.com/europe/equities/market_share/index/all/'
page = requests.get(url)
html = bs(page.text, 'lxml')
total_volume = html.findAll('td', class_='idx_val')
print(total_volume[645].text)

Output:
€4,378,517,621

【讨论】：