使用 Selenium 抓取 AliExpress 运费 - Python

Question

我正在尝试从 AliExpress 抓取特定商店的运输数据。但是，代码停在安道尔。此外，它会两次打印阿富汗并列出其运费，但该特定商店的第一个运往国家是阿根廷。我正在尝试将该数据存储到 excel 表中，但将其打印到控制台只是为了检查它是否正常工作。我的问题是为什么，它是否打印了两次阿富汗（也错误地打印了运费）以及为什么它停在安道尔。

以下链接中的国家/地区列表。它包括200多个国家。它不应该停留在安道尔。 https://www.dropbox.com/s/un7fh6pl3hc3567/Countries.txt?dl=0

运行程序后的结果：

阿富汗

07/25   US $7.37        AliExpress Standard Shipping        07/29   US $40.17       EMS Afghanistan

此供应商/运输公司不向您选择的国家/地区发货。

奥兰群岛

此供应商/运输公司不向您选择的国家/地区发货。

阿尔巴尼亚

此供应商/运输公司不向您选择的国家/地区发货。

奥尔德尼

此供应商/运输公司不向您选择的国家/地区发货。

阿尔及利亚

此供应商/运输公司不向您选择的国家/地区发货。

美属萨摩亚

此供应商/运输公司不向您选择的国家/地区发货。

安道尔

此供应商/运输公司不向您选择的国家/地区发货。

请参阅下面的代码：

def login(self):

    fail = True
    while fail:
        try:
            # Go to AliExpress Store.
            self.driver.get("https://www.aliexpress.com/store/all-wholesale-products/5795287.html?spm=a2g0o.detail.1000061.2.266e75e9QrlB4U")
            time.sleep(3.5)
            # Change location to USA, some stores don't ship worldwide so no products will be displayed.
            self.driver.find_element_by_xpath("//span[contains(@class,'currency') and contains(text(),'USD')]").click()
            self.driver.find_element_by_xpath("//*[@id='nav-global']/div[6]/div/div/div[1]/div/a[1]").click()
            self.driver.find_element_by_xpath("//span[contains(@class,'shipping-text') and contains(text(),'United States')]").click()
            self.driver.find_element_by_xpath("//*[@id='nav-global']/div[6]/div/div/div[3]/button").click()
            time.sleep(1.5)
            # Click first image
            self.driver.find_element_by_class_name("picCore.lazy-load").click()
            time.sleep(2.5)
            self.driver.find_element_by_class_name("next-dialog-close").click()
            time.sleep(4)
            # locate shipping URL
            self.driver.find_element_by_xpath("//*[@id='root']/div/div[2]/div/div[2]/div[10]/span[1]").click()
            time.sleep(1)
            # Exit while loop
            fail = False

        except Exception:
            pass

    try:
        # List of countries (used by AliExpress) stored in a text file.
        # Create a list and store all the countries in a variable called "countries".
        with open('Countries.txt', 'r') as f:
            countries = [line.strip() for line in f]

        # Loop through the countries and get shipping prices.
        for country in countries:
            # Click dropdown arrow to access the countries and use variable "countries" to located each country.
            self.driver.find_element_by_class_name("next-select-arrow").click()
            self.driver.find_element_by_xpath("//span[contains(@class,'country-name') and contains(text(),'"+country+"')]").click()
            print(country)
            try:
                # If supplier don't ship to this country locate and print.
                # "This Supplier/Shipping Company does not deliver to your selected Country/Region."
                elements = self.driver.find_element_by_class_name("next-message-content")
                print(elements.text)

            except Exception:

                try:
                    # If supplier does ship to this country locate and print cost, tracking and carrier.
                    elements = self.driver.find_elements_by_class_name('table-td')
                    for e in elements:
                        print(e.text + "\t", end='')

                except Exception:
                    print("Fail!!")

            time.sleep(2.5)

    except Exception:
        pass

'''

Answer 1

我认为您的代码无法正常工作，因为该行

self.driver.find_element_by_xpath("//span[contains(@class,'country-name') and contains(text(),'"+country+"')]").click()

总是抛出异常。原因？你要求你的代码点击他看不到的东西。

所以我这样做了：

self.driver.find_element_by_xpath('//*[@id="ae-search-select-1"]').send_keys(country)
self.driver.find_element_by_xpath('/html/body/div[12]/div[2]/div/div[1]/div/div[2]/span/div/div/div/ul/li').click()

你只需要用新的来改变旧的行。

它有什么作用？ 它在搜索栏中输入国家名称并点击结果。

--- 编辑 ---

try:
   with open('Countries.txt', 'r') as f:
      countries = [line.strip() for line in f]
   for country in countries:
       print(country)
       self.driver.find_element_by_class_name("next-select-arrow").click()
       self.driver.find_element_by_xpath('//*[@id="ae-search-select-1"]').send_keys(country)

       self.driver.find_elements_by_class_name("next-menu-item")[-1].click()

       time.sleep(1)

          try:
              elements = self.driver.find_element_by_class_name("next-message-content")
              print(elements.text)

          except Exception:

               try:
                   elements = self.driver.find_elements_by_class_name('table-tr')
                   for i in range(len(elements)):
                      print(elements[i].text.replace("\n",'\t'))
                except Exception:
                    print("Fail!!")

          elements = ""

 except Exception:
    pass

所以这是新代码，它与国家/地区的价格相匹配