尝试使用 BeautifulSoup 从 <td></td> 标记中检索文本

Question

所以我使用 BeautifulSoup 来抓取代码中的链接。艺术家姓名和链接都很好，但我不确定如何在第二个标签中访问国籍。

代码如下：

import requests
import csv
from bs4 import BeautifulSoup

def findName():
  page = requests.get('https://web.archive.org/web/20121007172955/https://www.nga.gov/collection/anB1.htm')

  soup = BeautifulSoup(page.text, 'html.parser')

  last_links = soup.find(class_='AlphaNav')
  last_links.decompose()

  f = csv.writer(open('h-artist_lastname.csv', 'w')) # Create a file to write
  f.writerow(['Last Name, First Name', 'Nationality', 'Link'])

  artist_name_list = soup.find(class_='BodyText') 
  artist_name_list_items = artist_name_list.find_all('a') 
  artist_nationality_list_items = artist_name_list.find_all('td')

  print(artist_nationality_list_items)

  for artist_name in artist_name_list_items:        
    names = artist_name.contents[0]
    #nationalities = artist_nationality_list_items.contents[0]  
    links = 'https://web.archive.org' + artist_name.get('href')

    #print(nationalities)

    f.writerow([names, links])

findName()

如果我取消注释 for 循环中的行，我会得到一个运行时错误，这是我所期望的。 print 语句为我提供了这个 Artist_nationality_list_items 的值：

<td><a href="/web/20121007172915/http://www.nga.gov/cgi-bin/tsearch?artistid=32727">Babbitt, Platt D.</a></td>, <td>American, died 1879</td>, ..... <- follows this pattern for every artist

基本上，我想要“美国人，于 1879 年去世”的角色。

Answer 1

您可以使用select 接受带有:nth-child() 的CSS 选择器来在每个<tr> 中选择第二个<td> 而不是find_all，所以这样：

artist_nationality_list_items = artist_name_list.find_all('td')

变成：

artist_nationality_list_items = artist_name_list.select('td:nth-child(2)')

Answer 2

您仍然可以使用 contents，但不要被所有列表所困扰 - 选择更具体的目标并更流畅地获取所有信息。

会发生什么？

您将artist_nationality_list_items（一个列表）视为单个元素，这是行不通的。

如何解决？

要从您的 artist_nationality_list_items 获得正确的结果，您也必须对其进行迭代。

（有效，但不是个好主意）：

for i,artist_name in enumerate(artist_name_list_items):        
    names = artist_name.contents[0]
    nationalities = artist_nationality_list_items[i+1].contents[0]  
    links = 'https://web.archive.org' + artist_name.get('href') 

另一种更精简的方法

import requests, csv
from bs4 import BeautifulSoup

def findName():
    page = requests.get('https://web.archive.org/web/20121007172955/https://www.nga.gov/collection/anB1.htm')

    soup = BeautifulSoup(page.text, 'html.parser')

    f = csv.writer(open('h-artist_lastname.csv', 'w')) # Create a file to write
    f.writerow(['Last Name, First Name', 'Nationality', 'Link'])
    
    for row in soup.select('div.BodyText h3+table tr'):

        names = row.contents[0].text
        nationalities = row.contents[1].text
        links = 'https://web.archive.org' + row.a.get('href')

        #print([names,nationalities,links])

        f.writerow([names,nationalities,links])

findName()

Answer 3

一些草率的解决方法有点拙劣的答案，但这导致了我所需要的：

import requests
import csv
from bs4 import BeautifulSoup

def findName():
  page = requests.get('https://web.archive.org/web/20121007172955/https://www.nga.gov/collection/anB1.htm')

  soup = BeautifulSoup(page.text, 'html.parser')

  last_links = soup.find(class_='AlphaNav')
  last_links.decompose()

  f = csv.writer(open('b-artist_lastname.csv', 'w')) # Create a file to write
  f.writerow(['Last Name, First Name', 'Nationality', 'Link'])

  artist_name_list = soup.find(class_='BodyText') 
  artist_name_list_items = artist_name_list.find_all('a') 

  i = 2

  for artist_name in artist_name_list_items:   
    str_list = list('td:nth-of-type(i)')
    str_list[15] = str(i)

    selection = "".join(str_list)

    names = artist_name.contents[0]
    nationality = artist_name_list.select(selection)  
    links = 'https://web.archive.org' + artist_name.get('href')

    nat_to_str = str(nationality)
    nat_str_final = nat_to_str[5:len(nat_to_str) - 6]

    #print(nat_str_final)

    f.writerow([names, nat_str_final, links])
    i += 2

findName()

感谢所有回答的人。使用 'td:nth-of-type()' 似乎可行，但对我来说，要让每个艺术家都出现在页面上，我需要每次都增加 nth-of-type 内部的值，所以我使用了一个字符列表和在每次遍历时增加 I 后将它们转换为字符串。