【问题标题】:Python/Selenium - How to process extracted data from App storePython/Selenium - 如何处理从应用商店提取的数据
【发布时间】:2020-09-14 23:11:08
【问题描述】:

我正在使用 Selenium/Python 来解析来自 Apple 应用商店的评论。我使用以下代码提取前五条评论的数据:

网址:https://apps.apple.com/us/app/lemonade-insurance/id1055653645#see-all/reviews

wait = WebDriverWait(driver, 5)
response_ratings = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".we-customer-review")))

response_container = []

for e in response_ratings[:5]:
    response_container.append(e.get_attribute('innerHTML'))

print(response_container[0])

然后,我打印第一个输出

我希望在第一次审核时获得星号5 out of 5、日期July 6, 2019、标题Convenient and Affordable!!!!、评论The Lemonade app is so easy to use as well as having affordable rates!... 和开发者回复Thanks so much for your awesome review!! We're so happy to have you in...

我如何获得上述信息?提前感谢您的帮助

【问题讨论】:

    标签: python-3.x selenium-webdriver web-scraping


    【解决方案1】:

    您可以使用BeautifulSoup 解析innerHTML 并得到您要查找的内容。

    一种方法是:

    import re
    
    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    
    from bs4 import BeautifulSoup
    
    
    link = 'https://apps.apple.com/us/app/lemonade-insurance/id1055653645#see-all/reviews'
    stars = re.compile(r"\d out of \d")
    
    with webdriver.Chrome() as driver:
        wait = WebDriverWait(driver, 10)
        driver.get(link)
        elements = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".we-customer-review")))
    
        for elem in elements:
            s = BeautifulSoup(elem.get_attribute("innerHTML"), "html.parser")
            review_date = s.find("time").text
            review_body = s.find("p").text
            review_title = s.find("h3", {"data-test-customer-review-title": ""}).text.strip()
            review_stars = ''.join(re.findall(stars, str(s.find("figure"))))
            dev_response = s.find_all("p", {"data-test-bidi": ""})
            print(f"{review_title} | {review_date} | {review_stars}")
            print(review_body)
            print(dev_response[1].text if len(dev_response) > 1 else "")
            print("-" * 80)
    

    打印出来:

    Convenient and Affordable!!!! | 07/06/2019 | 5 out of 5
    The Lemonade app is so easy to use as well as having affordable rates! It took me all of 15 minutes to sign-up, pick a coverage and a deductible. Very nice customer service as well as very informative. At anytime of any day of the week I can log onto my account and check everything as well as make any necessary changes that I may need or want. The chat option feature works fantastic! Whenever I have a question I just go to the chat feature and within seconds someone is there to help and answer all my questions regarding their services and my plan coverage. I wished they had a referral feature cause I’ve already set up a couple of family members with the company as well. They were amazed that it only took about 10-15 minutes to setup and just how affordable it is!! I’ve gotten quotes from so many other companies but the monthly payments and deductible were too expensive, I was a little hesitant at first but I said hey I should at least give it a try and so far so good!!! I’m hoping that it’ll never come to the point where I’ll actually need to file a claim but if so I feel confident that the process will be easy and stress free considering how much stress I’m going to actually have due to a burglary or theft. I have faith that we’re going to have a very long relationship. Thanks to all the developers of the Lemonade App, the name of the company is nice too!!!!
    Thanks so much for your awesome review!! We're so happy to have you in our Lemonade community!
    --------------------------------------------------------------------------------
    Made the best lemonade I’ve ever had! | 07/01/2018 | 5 out of 5
    I have been telling EVERYONE about Lemonade. I don’t know how, but, getting insurance through your company is actually FUN!  I have never had so much FUN doing a chore that typically involves a boring Q & A.  The app really made me feel like I was getting insurance through a friend. I smiled with the shout out to my horoscope sign after entering my birthday, I loved the “making lemonade” process when getting the quote, and (because I have a future date for the policy to start) I absolutely adore the countdown.  I look at it almost daily and become even more excited for my move (and I really don’t like having to move so this is really helping).  The use of unclaimed funds going to charity actually makes my heart melt.  By providing freedom to choose what organization you would like to contribute to truly makes me feel like I am giving back in some way, and it is beyond noble and inspiring for you to use that money to help others turn lemons into lemonade.  Also, your customer service has been impeccable.  Every question I have had has been answered quickly and by a friendly representative of the company.  I don’t know much about the insurance world, other than we need to have it, but, you make me want to work for you!!!  Where do I sign up?
    
    --------------------------------------------------------------------------------
    

    【讨论】:

    • 非常感谢先生,很高兴学习使用 bs4 可以完成这项工作。
    猜你喜欢
    • 2021-11-15
    • 1970-01-01
    • 2015-02-14
    • 1970-01-01
    • 1970-01-01
    • 2016-09-01
    • 2018-09-27
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多