选择性解析 BeautifulSoup

Question

我想解析来自 Drug 网站的数据。这个解析需要有选择性，这是我使用的代码：

import requests
from bs4 import BeautifulSoup

def get_details(url):
    print('details:', url)

    # get subpage
    r = requests.get(url)
    soup = BeautifulSoup(r.text ,"lxml")

    # get data on subpabe
    dts = soup.findAll('dt')
    dds = soup.findAll('dd')

    # display details
    for dt, dd in zip(dts, dds):
        print(dt.text)
        print(dd.text)
        print('---')

    print('---------------------------')

def drug_data():
    url = 'https://www.drugbank.ca/drugs/'

    while url:
        print(url)
        r = requests.get(url)
        soup = BeautifulSoup(r.text ,"lxml")

        # get links to subpages
        links = soup.select('strong a')
        for link in links:
            # exeecute function to get subpage
            get_details('https://www.drugbank.ca' + link['href'])

        # next page url
        url = soup.findAll('a', {'class': 'page-link', 'rel': 'next'})
        print(url)
        if url:
            url = 'https://www.drugbank.ca' + url[0].get('href')
        else:
            break

drug_data()

这运作良好。但是更深入和选择性的解析呢？假设这种药物：https://www.drugbank.ca/drugs/DB01614 当我使用我的代码解析“PATENT”时，它会将“PATENT”的所有信息（表示为子表）连接到一个段落中。

理想情况下，如果我可以解析专利但只提取“专利号”、“批准”和标志所代表的国家！在单独的列中！ 一些帮助 ？

这是专利截图： enter image description here

Answer 1

如果您正在寻找Accession Number 和Groups，您可以执行以下操作：

def get_details(url):
    print('Details:', url)

    r = requests.get(url)
    soup = BeautifulSoup(r.text, 'html.parser')

    accession_dt = soup.find('dt', text='Accession Number')
    accession_number = accession_dt.nextSibling.string
    groups_dt = soup.find('dt', text='Groups')
    groups = groups_dt.nextSibling.string

    print('Accession number: ' + accession_number)
    print('Groups: ' + groups)

对于你提供的url，输出如下：

>>> get_details('https://www.drugbank.ca/drugs/DB01614')
Details: https://www.drugbank.ca/drugs/DB01614
Accession number: DB01614
Groups: Approved, Vet Approved

---

如果你想概括这一点，你可以定义一个函数，返回作为参数传递的键的文本：

def get_value(soup, key):
    key_dt = soup.find('dt', text=key)
    return key_dt.nextSibling.string

要使用这个功能，你可以这样做：

def get_details(url):
    print('Details:', url)
    r = requests.get(url)
    soup = BeautifulSoup(r.text, 'html.parser')
    accession_number = get_value(soup, 'Accession Number')
    groups = get_value(soup, 'Groups')
    print('Accession number: ' + accession_number)
    print('Groups: ' + groups)

这给出了与上图相同的输出。

---

编辑：问题的答案

这将直接给出你想要的。

def get_details(url):
    print('Details:', url)

    r = requests.get(url)
    soup = BeautifulSoup(r.text, 'html.parser')

    patents = soup.find('dt', text='Patents').nextSibling
    if patents.string == 'Not Available':
        print('Patent: Not Available')
    else:
        for i, row in enumerate(patents.find('tbody').findAll('tr')):
            print('\nPatent entry %d:' % (i+1))
            patent_number = row.find('a').text
            patent_approved = row.findAll('td')[2].text
            patent_country = row.find('img')['alt']
            print('Patent number: ' + patent_number)
            print('Approved: ' + patent_approved)
            print('Country: ' + patent_country)

对于药物：https://www.drugbank.ca/drugs/DB00639，输出为

Details: https://www.drugbank.ca/drugs/DB00639

Patent entry 1:
Patent number: US5266329
Approved: 1993-11-30
Country: Us

Patent entry 2:
Patent number: US5993856
Approved: 1997-11-17
Country: Us