从多个同名跨度元素中刮取跨度文本（IMDB）

Question

for year_url in years_url:

    # For every page in the interval 1-4
    for page in pages:

        # Make a get request
        response = requests.get('http://www.imdb.com/search/title?release_date=' + year_url +
        '&sort=num_votes,desc&page=' + page)

        # Parse the content of the request with BeautifulSoup
        page_html = BeautifulSoup(response.text, 'lxml')

        # Select all the 50 movie containers from a single page
        mv_containers = page_html.find_all('div', class_ = 'lister-item mode-advanced')

        # For every movie of these 50
        for container in mv_containers:
            # If the movie has a Metascore, then:
            if container.find('div', class_ = 'ratings-metascore') is not None:

                # Scrape the name
                name = container.h3.a.text
                names.append(name)

                # Scrape the genre
                genre = container.p.find('span', class_ = 'genre').text.rstrip().replace("\n","").split(",")
                genres.append(genre)


                # Scrape the runtime
                runtime = container.p.find('span', class_ = 'runtime').text
                runtimes.append(runtime)


                # Scrape the year
                year = container.h3.find('span', class_ = 'lister-item-year').text
                years.append(year)

                # Scrape the IMDB rating
                imdb = float(container.strong.text)
                imdb_ratings.append(imdb)

                # Scrape the Metascore
                m_score = container.find('span', class_ = 'metascore').text
                metascores.append(int(m_score))

                # Scrape the number of votes
                vote = container.find('span', attrs = {'name':'nv'})['data-value']
                votes.append(int(vote))

                # Scrape the GrossMill
                gross = int(container.find('span', text='Gross:').find_next('span')['data-value'].replace(',', ''))
                print(gross)
                grossmill.append(gross)

我无法从上述网址创建总收藏数据 vote 和 Gross 具有相同的属性，所以我发现很难从下面提到的链接中提取总数据。我怎么能提取选票。 链接到 url = "https://www.imdb.com/search/title/?release_date=2019&sort=num_votes,desc&page=1"

Answer 1

您的错误意味着对于给定的电影，没有找到 span 和 text='Gross'。正如我从您正在抓取的IMDB webpage 中看到的那样，碰巧有些电影虽然有 Metascore，但没有显示它们的总分。电影1917就是这样。

在使用方法find_next() 调用它之前，您应该首先检查gross 的存在。

替换：

gross = int(container.find('span', text='Gross:').find_next('span')['data-value'].replace(',', ''))

作者：

gross = container.find('span', text='Gross:')
if gross:
    gross = int(gross.find_next('span')['data-value'].replace(',', ''))