BeautifulSoup 用“N/A”填充缺失的信息不起作用答案

【问题标题】：BeautifulSoup filling missing information with "N/A" not workingBeautifulSoup 用“N/A”填充缺失的信息不起作用
【发布时间】：2017-07-05 04:41:48
【问题描述】：

我正在以下网站练习我的网页抓取技巧：“http://web.californiacraftbeer.com/Brewery-Member”

我到目前为止的代码如下。似乎我得到了正确的公司计数，但我的 CSV 文件中出现了重复的行，我认为每当公司缺少信息时就会出现这种情况。在我的代码的多个部分中，我试图检测并用文本“N/A”替换缺失的信息，但它不起作用。我猜这个问题可能与 Zip() 函数有关，但我不确定如何解决。

非常感谢任何帮助！

"""
Grabs brewery name, contact person, phone number, website address, and email address 
for each brewery listed on the website.
"""

import requests, csv
from bs4 import BeautifulSoup

url = "http://web.californiacraftbeer.com/Brewery-Member"
res = requests.get(url)
soup = BeautifulSoup(res.content, "lxml")
each_company = soup.find_all("div", {"class": "ListingResults_All_CONTAINER ListingResults_Level3_CONTAINER"})
error_msg = "N/A" 

def scraper():
    """Grabs information and writes to CSV"""
    print("Running...")
    results = []
    count = 0

    for info in each_company:
        try:
            company_name = info.find_all("span", itemprop="name")
        except Exception as e:
            company_name = "N/A"
        try:
            contact_name = info.find_all("div", {"class": "ListingResults_Level3_MAINCONTACT"})
        except Exception as e:
            contact_name = "N/A"
        try:
            phone_number = info.find_all("div", {"class": "ListingResults_Level3_PHONE1"})
        except Exception as e:
            phone_number = "N/A"
        try:
            website = info.find_all("span", {"class": "ListingResults_Level3_VISITSITE"})
        except Exception as e:
            website = "N/A"

        for company, contact, phone, site in zip(company_name, contact_name, phone_number, website):
            count += 1
            print("Grabbing {0} ({1})...".format(company.text, count))
            newrow = []
            try:
                newrow.append(company.text)
            except Exception as e:
                newrow.append(error_msg)
            try:
                newrow.append(contact.text)
            except Exception as e:
                newrow.append(error_msg)
            try:
                newrow.append(phone.text)
            except Exception as e:
                newrow.append(error_msg)
            try:
                newrow.append(site.find('a')['href'])
            except Exception as e:
                newrow.append(error_msg)
            try:
                newrow.append("info@" + company.text.replace(" ", "").lower() + ".com")
            except Exception as e:
                newrow.append(error_msg)
        results.append(newrow)

    print("Done")
    outFile = open("brewery.csv", "w")
    out = csv.writer(outFile, delimiter=',',quoting=csv.QUOTE_ALL, lineterminator='\n')
    out.writerows(results)
    outFile.close()

def main():
    """Runs web scraper"""
    scraper()

if __name__ == '__main__':
    main()

【问题讨论】：

标签： python csv beautifulsoup

【解决方案1】：

来自bs4 docs

"如果 find_all() 找不到任何东西，它返回一个空列表。如果 find() 什么都找不到，它返回 None"

因此，例如，当 company_name = info.find_all("span", itemprop="name") 运行并且不匹配任何内容时，它不会引发异常并且 "NA" 永远不会被设置。

此时需要检查company_name是否为空列表：

if not company_name:
    company_name = "N/A"

【讨论】：