Selenium 中的 Python IndexError 和 UnboundLocalError

Question

抱歉，如果这是一个基本问题，因为我对 python 还比较陌生。我正在尝试使用 Selenium 开发一个网页抓取脚本，并且已经完成了几乎所有必要的功能（从一个页面导航到另一个页面，在一个页面中定位和打开所有 url）。

但是，由于我试图抓取的网站的性质，一些元素在某些页面上被省略，而在其他页面上却存在。在它们被省略的情况下，终端返回一个IndexError，我目前正在使用异常绕过它。

但是，当我尝试打印抓取的数据时，我收到以下错误：

UnboundLocalError: local variable 'manufacturer' referenced before assignment

我知道这可能是因为我为 IndexError 设置了一个异常，然后在随后的 print 命令中引用了它。

我是否仍然可以通过规避这两个问题来抓取我正在寻找的信息，如果是这样，我将如何将抓取的数据导出到 csv 文件中？

我的函数代码如下：

def scrape():
    browser.implicitly_wait(7)
    try:
        collection = browser.find_elements(By.XPATH,'//*[@id="page-content-wrapper"]/div/ul/li[5]/a')[0].text
        description = browser.find_elements(By.XPATH,'//*[(@id = "child-1")]//p')[0].text
        dimension = browser.find_elements(By.XPATH,'//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 1) and parent::*)]//p')[0].text
        finish = browser.find_elements(By.XPATH,'//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 2) and parent::*)]//p')[0].text
        country = browser.find_elements(By.XPATH,'//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 3) and parent::*)]//p')[0].text
        manufacturer = browser.find_elements(By.XPATH,'//div[(((count(preceding-sibling::*) + 1) = 4) and parent::*)]//p')[0].text
    except IndexError:
        pass
    print(collection, description, dimension, finish, country, manufacturer)
    browser.back()

非常感谢！

Answer 1

由于您的变量是在 try 块中创建的，如果其中一个失败，则不会创建该变量以及该变量下方的任何内容，并且当您尝试引用它时，解释器不知道该变量应该是什么是。如果您执行以下操作，打印语句将起作用：

def scrape():
    browser.implicitly_wait(7)
    collection = ""
    description = ""
    dimension = ""
    finish = ""
    country = ""
    manufacturer = ""

    try:
        collection = browser.find_elements(By.XPATH,'//*[@id="page-content-wrapper"]/div/ul/li[5]/a')[0].text
        description = browser.find_elements(By.XPATH,'//*[(@id = "child-1")]//p')[0].text
        dimension = browser.find_elements(By.XPATH,'//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 1) and parent::*)]//p')[0].text
        finish = browser.find_elements(By.XPATH,'//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 2) and parent::*)]//p')[0].text
        country = browser.find_elements(By.XPATH,'//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 3) and parent::*)]//p')[0].text
        manufacturer = browser.find_elements(By.XPATH,'//div[(((count(preceding-sibling::*) + 1) = 4) and parent::*)]//p')[0].text
    except IndexError:
        pass
    print(collection, description, dimension, finish, country, manufacturer)
    browser.back()

所以现在你必须处理失败变量下面的变量也没有被分配的问题；我建议使用字典：

def scrape():
    browser.implicitly_wait(7)
    page_elements = dict()
    page_elements['collection'] = ""
    page_elements['description'] = ""
    page_elements['dimension'] = ""
    page_elements['finish'] = ""
    page_elements['country'] = ""
    page_elements['manufacturer'] = ""
    try:
        collection = browser.find_elements(By.XPATH,'//*[@id="page-content-wrapper"]/div/ul/li[5]/a')[0].text
    except IndexError:
        pass
    try:
        description = browser.find_elements(By.XPATH,'//*[(@id = "child-1")]//p')[0].text
    except IndexError:
        pass
    try:
        dimension = browser.find_elements(By.XPATH,'//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 1) and parent::*)]//p')[0].text
    except IndexError:
        pass
    try:
        finish = browser.find_elements(By.XPATH,'//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 2) and parent::*)]//p')[0].text
    except IndexError:
        pass
    try:
        country = browser.find_elements(By.XPATH,'//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 3) and parent::*)]//p')[0].text
    except IndexError:
        pass
    try:
        manufacturer = browser.find_elements(By.XPATH,'//div[(((count(preceding-sibling::*) + 1) = 4) and parent::*)]//p')[0].text
    except IndexError:
        pass
    print(page_elements)
    browser.back()

再一次，为每个语句使用 try 块有点麻烦，所以如果你想将路径保存在不同的字典中，你可以这样做：

def scrape():
    browser.implicityl_wait(7)
    page_elements = dict()
    page_elements['collection'] = ""
    page_elements['description'] = ""
    page_elements['dimension'] = ""
    page_elements['finish'] = ""
    page_elements['country'] = ""
    page_elements['manufacturer'] = ""
    
    element_paths = dict()
    element_paths['collection'] = '//*[@id="page-content-wrapper"]/div/ul/li[5]/a'
    element_paths['description'] = '//*[(@id = "child-1")]//p'
    element_paths['dimension'] = '//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 1) and parent::*)]//p'
    element_paths['finish'] = '//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 2) and parent::*)]//p'
    element_paths['country'] = '//*[(@id = "detailed-description")]//div[(((count(preceding-sibling::*) + 1) = 3) and parent::*)]//p'
    element_paths['manufacturer'] = '//div[(((count(preceding-sibling::*) + 1) = 4) and parent::*)]//p'

    for element in page_elements:
        try:
            page_elements[element] = browser.find_elements(By.XPATH, element_paths[element])[0].text
        except IndexError:
            pass
    print(page_elements)
    browser.back()