用 Beautiful Soup 解析链接 URL

Question

我正在使用漂亮的汤 (BS4) 和 python 通过 waybackmachine/webarchive 从黄页中抓取数据。我可以轻松返回企业名称和电话号码，但是当我尝试检索企业的网站 url 时，我只返回整个 div 标签。

#Import Dependencies
from splinter import Browser
from bs4 import BeautifulSoup 
import requests
import pandas as pd 

# Path to chromedriver
!which chromedriver 

# Set the executable path and initialize the chrome browser in splinter
executable_path = {'executable_path': '/usr/local/bin/chromedriver'}
browser = Browser('chrome', **executable_path) 

#visit Webpage 
url = 'https://web.archive.org/web/20171004082203/https://www.yellowpages.com/houston-tx/air-conditioning-service-repair'
browser.visit(url) 

# Convert the browser html to a soup object and then quit the browser
html = browser.html
soup = BeautifulSoup(html, "html.parser")  

##Scrapers
#business name
print(soup.find('a', class_='business-name').text)
#Telephone
print(soup.find('li', class_='phone primary').text)
#website
print(soup.find('div', class_='links'))

我怎样才能只返回公司的网站 URL？谢谢。

Answer 1

改为返回href：

print(soup.find('a', class_='business-name')['href'])

Answer 2

你可以做这样的工作：

1. 获取所有链接列表，然后获取索引0值
2. 然后用分隔符分割它：“http://”

检查下面的更新代码：

#Import Dependencies
from splinter import Browser
from bs4 import BeautifulSoup 
import requests
import pandas as pd 

# Path to chromedriver
!which chromedriver 

# Set the executable path and initialize the chrome browser in splinter
executable_path = {'executable_path': '/usr/local/bin/chromedriver'}
browser = Browser('chrome', **executable_path) 

#visit Webpage 
url = 'https://web.archive.org/web/20171004082203/https://www.yellowpages.com/houston-tx/air-conditioning-service-repair'
browser.visit(url) 

# Convert the browser html to a soup object and then quit the browser
html = browser.html
soup = BeautifulSoup(html, "html.parser")  

##Scrapers
#business name
print(soup.find('a', class_='business-name').text)
#Telephone
print(soup.find('li', class_='phone primary').text)
#website
links = soup.find('div', class_='links').findAll("a")
originalLink = links[0].get("href").split("http://")[1]