无法使用 BeautifulSoup 从 select_one 中获取文本

Question

我正在尝试从下面的 HTML 解析时间，但无法使用 get_text 和 select_one 来提取 <time class = "published-date relative-date" ... /time> 内的 data-published-date 或 datetime。

<div class="content">
       <header>
        <h3 class="article-name">
         Curious Kids: Why is the Moon Called the Moon?
        </h3>
        <p class="byline">
         <span class="by-author">
          By
          <span style="white-space:nowrap">
           Toby Brown
          </span>
         </span>
         <time class="published-date relative-date" data-published-date="2019-12-13T12:00:28Z" datetime="2019-12-13T12:00:28Z">
         </time>
        </p>
       </header>

使用：

import requests
from bs4 import BeautifulSoup
url = 'https://www.space.com/news'
response = requests.get(url)
data = response.text
soup = BeautifulSoup(data, 'html.parser')

contents = soup.select('.content')
headlines = []
for item in contents:
  h_line = item.select_one('.article-name').get_text()
  author = item.select_one('.byline > span:nth-of-type(1) > span:nth-of-type(1)').get_text().strip()
  synopsis = item.select_one('.synopsis').get_text().strip() 
  date = item.select_one('.byline > time').get_text() 
  newsline = {'Headline': h_line, 'Author': author, 'Synopsis': synopsis, 'Date': dates}
  headlines.append(newsline) 

for line in headlines:   
  print(line)  

产生一个回溯错误，声称它是“NoneType”。此外，答案只能使用 BeautifulSoup 解析，不能使用 RegEx。

***更新： 我修改了答案以便能够在我的 for 循环中使用（这样我就可以遍历所有标题的源代码）

import requests
from bs4 import BeautifulSoup
url = 'https://www.space.com/news'
response = requests.get(url)
data = response.text
soup = BeautifulSoup(data, 'html.parser')

contents = soup.select('.content')
headlines = []
for item in contents:
  h_line = item.select_one('.article-name').get_text()
  author = item.select_one('.byline > span:nth-of-type(1) > span:nth-of-type(1)').get_text().strip()
  synopsis = item.select_one('.synopsis').get_text().strip() 
  dates = item.select_one('time').get('data-published-date')
  newsline = {'Headline': h_line, 'Author': author, 'Synopsis': synopsis, 'Date & Time Published': dates}
  headlines.append(newsline) 

for line in headlines:   
  print(line)   

Answer 1

from bs4 import BeautifulSoup
data = """
<div class="content">
       <header>
        <h3 class="article-name">
         Curious Kids: Why is the Moon Called the Moon?
        </h3>
        <p class="byline">
         <span class="by-author">
          By
          <span style="white-space:nowrap">
           Toby Brown
          </span>
         </span>
         <time class="published-date relative-date" data-published-date="2019-12-13T12:00:28Z" datetime="2019-12-13T12:00:28Z">
         </time>
        </p>
       </header>
"""


soup = BeautifulSoup(data, 'html.parser')

for item in soup.findAll('time', {'class': 'published-date relative-date'}):
    print(item.get('data-published-date'))

输出：

2019-12-13T12:00:28Z

深度版：

import requests
from bs4 import BeautifulSoup

r = requests.get('https://www.space.com/news')
soup = BeautifulSoup(r.text, 'html.parser')

headline = []
author = []
syn = []
time = []
for item in soup.findAll('h3', {'class': 'article-name'}):
    headline.append(item.text)
for item in soup.findAll('span', {'style': 'white-space:nowrap'}):
    author.append(item.get_text(strip=True))
for item in soup.findAll('p', {'class': 'synopsis'}):
    syn.append(item.get_text(strip=True))
for item in soup.findAll('time', {'class': 'published-date relative-date'}):
    time.append(item.get('data-published-date'))

for item in zip(headline, author, syn, time):
    print(item)

Answer 2

这个解决方案应该更有效，因为它只循环一次。

import requests
from bs4 import BeautifulSoup

r = requests.get('https://www.space.com/news')
soup = BeautifulSoup(r.text, 'html.parser')
contents = soup.find_all('div', {'class': 'content'})

headlines = []
for content in contents:
    h_line = content.h3.text
    author = content.span.text.strip('\n').split('\n\n')[-1]
    synopsis = content.find('p', {'class': 'synopsis'}).text.strip('\n')
    dates = content.time['data-published-date']
    newsline = {'Headline': h_line, 'Author': author, 'Synopsis': synopsis, 'Date & Time Published': dates}
    headlines.append(newsline)

print(headlines[0])

{'Author': 'Doris Elin Urrutia ',
 'Date & Time Published': '2019-12-13T21:55:10Z',
 'Headline': 'Space Photos: The Most Amazing Images This Week!',
 'Synopsis': 'Here are our picks for the most amazing space photos of the week.'}