无法从网页以自定义方式获取输出

Question

我在 python 中结合 BeautiflSoup 编写了一个脚本，使用选择器从网页中解析电影名称及其相应的功能。当我执行我的脚本时，它会部分获取所需的项目。如何获取所有电影名称及其特征？

我试过这样：

import requests
from bs4 import BeautifulSoup
from itertools import zip_longest

with requests.Session() as session:
    r = session.get('https://yts.am/browse-movies')
    soup = BeautifulSoup(r.text,"lxml")
    items = {item.text:itm.text for item,itm in zip(soup.select(".browse-movie-title"),soup.select("figcaption h4"))}
    print(items)

我得到的结果如下：

{'Halloween H20: 20 Years Later': '5.7 / 10', 'Rabbit': 'Horror', and so on-----

我想这是因为 zip() 函数。但是，我导入了zip_longest()，这可能会奏效，但我无法使用它。

Html 元素，其中一个电影的一个这样的功能是：

<figcaption class="hidden-xs hidden-sm">
<span class="icon-star"></span>
<h4 class="rating">5.7 / 10</h4>
<h4>Horror</h4>
<h4>Thriller</h4>
<span class="button-green-download2-big">View Details</span>
</figcaption>

这是一部电影的相关html：

<div class="browse-movie-bottom">
<a href="https://yts.am/movie/halloween-h20-20-years-later-1998" class="browse-movie-title">Halloween H20: 20 Years Later</a>
<div class="browse-movie-year">1998</div>
</div>

单个电影的预期输出：

'Halloween H20: 20 Years Later': ['5.7 / 10','Horror','Thriller']

Answer 1

您正在一次选择所有元素。可能很难分组。 zip 也不是您要找的东西。只需遍历卡片即可。

import requests
from bs4 import BeautifulSoup

with requests.Session() as session:
    r = session.get('https://yts.am/browse-movies')
    soup = BeautifulSoup(r.text,"lxml")  

    for movie in soup.select("div.browse-movie-wrap"):
        title = movie.select_one('a.browse-movie-title').text
        details = [detail.text for detail in movie.select('h4')]
        print((title, details))

输出将是，

('Heavy Weights', ['6.7 / 10', 'Comedy', 'Drama'])
('Get Shorty', ['6.9 / 10', 'Comedy', 'Crime'])
('Fred Claus', ['5.6 / 10', 'Comedy', 'Family'])
("Free Willy: Escape from Pirate's Cove", ['5.2 / 10'])
('Halloween: Resurrection', ['4.1 / 10', 'Comedy', 'Horror'])
('Ant-Man and the Wasp', ['7.2 / 10', 'Action', 'Adventure'])
('Rabbit', ['6.2 / 10', 'Thriller'])
('Halloween H20: 20 Years Later', ['5.7 / 10', 'Horror', 'Thriller'])
("Madeline's Madeline", ['6.9 / 10'])
('Halloween 5', ['5.2 / 10'])
('Halloween: The Curse of Michael Myers', ['4.9 / 10', 'Action', 'Horror'])
('Deck the Halls', ['4.9 / 10', 'Comedy', 'Family'])
('Halloween 4: The Return of Michael Myers', ['5.9 / 10', 'Horror', 'Thriller'])
('Dark Horse', ['6 / 10', 'Action', 'Comedy'])
('Double Whammy', ['5.7 / 10', 'Comedy', 'Crime'])
('Beyond Borders', ['6.5 / 10', 'Adventure', 'Drama'])
('Dead Man Running', ['6 / 10', 'Action', 'Crime'])
('Cougar Hunting', ['3.7 / 10', 'Comedy', 'Romance'])
('Cabin Boy', ['5.2 / 10', 'Adventure', 'Comedy'])
('Illang: The Wolf Brigade', ['5.5 / 10', 'Action', 'Sci-Fi'])