有没有办法从父网页的多个页面解析数据？

Question

所以我一直去一个网站获取NDC代码https://ndclist.com/?s=Solifenacin，我需要获取10位数的NDC代码，但在当前网页上只​​有8位数的NDC代码如下图所示

所以我点击了带下划线的 NDC 代码。并得到这个网页。

因此，我将这 2 个 NDC 代码复制并粘贴到 Excel 表中，然后对我显示的第一个网页上的其余代码重复此过程。但是这个过程需要很多时间，并且想知道 Python 中是否有一个库可以为我复制和粘贴 10 位 NDC 代码或将它们存储在列表中，然后我可以在完成后打印列表第一页上有所有 8 位 NDC 代码。 BeautifulSoup 会起作用还是有更好的库来实现这个过程？

编辑

这是我拥有的代码，但是一旦我运行它，它就会返回一个 tr 和 None 对象。

url ='https://ndclist.com/?s=Trospium'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

all_data = []
for a in soup.select('[data-title="NDC"] a[href]'):
    link_url = a['href']
    print('Processin link {}...'.format(link_url))

    soup2 = BeautifulSoup(requests.get(link_url).content, 'html.parser')
    for b in soup2.select('#product-packages a'):
        link_url2 = b['href']
        print('Processing link {}... '.format(link_url2))
        soup3 = BeautifulSoup(requests.get(link_url2).content, 'html.parser')
        for link in soup3.findAll('tr', limit=7)[1]:
            print(link.name)
            all_data.append(link.name)

print('Trospium')
print(all_data)

Answer 1

是的，BeautifulSoup 在这种情况下是理想的。此脚本将从页面打印所有 10 位代码：

import requests
from bs4 import BeautifulSoup

url = 'https://ndclist.com/?s=Solifenacin'

soup = BeautifulSoup(requests.get(url).content, 'html.parser')

all_data = []
for a in soup.select('[data-title="NDC"] a[href]'):
    link_url = a['href']
    print('Processin link {}...'.format(link_url))

    soup2 = BeautifulSoup(requests.get(link_url).content, 'html.parser')
    for link in soup2.select('#product-packages a'):
        print(link.text)
        all_data.append(link.text)

# In all_data you have all codes, uncoment to print them:
# print(all_data)

打印：

Processin link https://ndclist.com/ndc/0093-5263...
0093-5263-56
0093-5263-98
Processin link https://ndclist.com/ndc/0093-5264...
0093-5264-56
0093-5264-98
Processin link https://ndclist.com/ndc/0591-3796...
0591-3796-19
Processin link https://ndclist.com/ndc/27241-037...
27241-037-03
27241-037-09

... and so on.

编辑：（我也得到描述的版本）：

import requests
from bs4 import BeautifulSoup

url = 'https://ndclist.com/?s=Solifenacin'

soup = BeautifulSoup(requests.get(url).content, 'html.parser')

all_data = []
for a in soup.select('[data-title="NDC"] a[href]'):
    link_url = a['href']
    print('Processin link {}...'.format(link_url))

    soup2 = BeautifulSoup(requests.get(link_url).content, 'html.parser')
    for code, desc in zip(soup2.select('a > h4'), soup2.select('a + p.gi-1x')):
        code = code.get_text(strip=True).split(maxsplit=1)[-1]
        desc = desc.get_text(strip=True).split(maxsplit=2)[-1]
        print(code, desc)
        all_data.append((code, desc))

# in all_data you have all codes:
# print(all_data)

打印：

Processin link https://ndclist.com/ndc/0093-5263...
0093-5263-56 30 TABLET, FILM COATED in 1 BOTTLE
0093-5263-98 90 TABLET, FILM COATED in 1 BOTTLE
Processin link https://ndclist.com/ndc/0093-5264...
0093-5264-56 30 TABLET, FILM COATED in 1 BOTTLE
0093-5264-98 90 TABLET, FILM COATED in 1 BOTTLE
Processin link https://ndclist.com/ndc/0591-3796...
0591-3796-19 90 TABLET, FILM COATED in 1 BOTTLE

...and so on.