如何使用 BeautifulSoup4 和 requests 获取标题的内容

Question

所以我从这个链接中取了药物的标题：Medicines List

现在我想获取每种药物的内容，同时每种药物都有它自己的链接 例子 ： Medicines Example

如何使用 BeautifulSoup4 和请求库获取该药物的内容？

import requests
from bs4 import BeautifulSoup
from pprint import pp

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0'
}


def main(url):
    r = requests.get(url, headers=headers)
    soup = BeautifulSoup(r.text, 'lxml')
    title = [x.text for x in soup.select(
        'a[class$=section__item-link]')]
    count = 0
    for x in range (0, len(title)):
        count += 1
        print("{0}. {1}\n".format(count, title[x]))


main('https://www.klikdokter.com/obat')

Answer 1

根据我所看到的来自https://www.klikdokter.com/obat 的响应，您应该能够执行以下操作：-

import requests
from bs4 import BeautifulSoup
AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 11_5_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Safari/605.1.15'
BASEURL = 'https://www.klikdokter.com/obat'
headers = {'User-Agent': AGENT}
response = requests.get(BASEURL, headers=headers)
response.raise_for_status()
soup = BeautifulSoup(response.text, 'html.parser')
for tag in soup.find_all('a', class_='topics-index--section__item-link'):
    href = tag.get('href')
    if href is not None:
        print(href)
        response = requests.get(href, headers=headers)
        response.raise_for_status()
        """ Do your processing here """