Requests-html：在烧瓶上运行时出错答案

【问题标题】：Requests-html: error while running on flaskRequests-html：在烧瓶上运行时出错
【发布时间】：2020-11-05 02:15:17
【问题描述】：

我准备了一个使用requests-html 的脚本，它运行良好。

我将它部署在烧瓶应用程序中，现在它给了我RuntimeError: There is no current event loop in thread 'Thread-3'.

这是完整的错误：

Traceback (most recent call last):
  File "C:\Users\intel\AppData\Local\Programs\Python\Python38\Lib\site-packages\flask\app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
 .
 .
 .
  File "C:\Users\intel\Desktop\One page\main.py", line 18, in hello_world
    r.html.render()
  File "C:\Users\intel\AppData\Local\Programs\Python\Python38\Lib\site-packages\requests_html.py", line 586, in render
    self.browser = self.session.browser  # Automatically create a event loop and browser
  File "C:\Users\intel\AppData\Local\Programs\Python\Python38\Lib\site-packages\requests_html.py", line 727, in browser
    self.loop = asyncio.get_event_loop()
  File "C:\Users\intel\AppData\Local\Programs\Python\Python38\Lib\asyncio\events.py", line 639, in get_event_loop

    raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'Thread-3'.

这是原始代码：

from flask import Flask
from requests_html import HTMLSession
from bs4 import BeautifulSoup


app = Flask(__name__)


@app.route('/<user>')
def hello_world(user):
    session = HTMLSession()
    r = session.get('https://medium.com/@' + str(user))

    print(r)

    r.html.render()

    divs = r.html.find('div')

    lst = []

    for div in divs:
        soup = BeautifulSoup(div.html, 'html5lib')
        div_tag = soup.find()
        try:
            title = div_tag.section.div.h1.a['href']
            if title not in lst:
                lst.append(title)
        except:
            pass

    return "\n".join(lst)


if __name__ == '__main__':
    app.run(debug=True)

【问题讨论】：

我认为flask 和request-html 不能很好地协同工作。见this SO post。

标签： python flask web-scraping beautifulsoup python-requests-html

【解决方案1】：

您应该使用在烧瓶之上构建的 Quart 库。这将解决您的问题。

安装：pip install quart

from quart import Quart
from requests_html import AsyncHTMLSession
app  = Quart(__name__)
@app.route('/<user>')
def hello_world(user):
    session = AsyncHTMLSession()
    r = await session.get('https://medium.com/@' + str(user))

    print(r)

    await r.html.arender()

    divs = r.html.find('div')

    lst = []

    for div in divs:
        soup = BeautifulSoup(div.html, 'html5lib')
        div_tag = soup.find()
        try:
            title = div_tag.section.div.h1.a['href']
            if title not in lst:
                lst.append(title)
        except:
            pass

【讨论】：