如何使用 Selenium 将 Python 字符串转换为 Pandas DataFrame？

Question

我仍在学习 Python，并且一直在练习网页抓取。我最近想尝试 Selenium。这是我目前的项目。

我的目标是将我的输出（我相信它是一个字符串）转换为 pandas 数据框。不幸的是，我不知道如何将所有数据分成各自的列。

到目前为止，这是我的代码：

from selenium import webdriver
import time
import pandas as pd

class FindByIdName():
    
    def test(self):
        baseUrl = 'https://vaccinefinder.org/'
        driver = webdriver.Chrome('C:\webdrivers\chromedriver.exe')
        driver.get(baseUrl)        
        time.sleep(3)
        
        #find covid vaccine button    
        vaccineElement = driver.find_element_by_xpath("/html//div[@id='__next']/div/div[2]/div[1]//button[.='Find COVID-19 Vaccines']")        
        vaccineElement.click()        
        time.sleep(3)  
        
        #uncheck moderna vaccine      
        modernaElement = driver.find_element_by_xpath("/html//div[@id='split-screen-content']/form/div[@class='search-form__fields-row']//label[.='Moderna COVID Vaccine']")
        modernaElement.click()
        time.sleep(2)
        
        #uncheck pfizer vaccine
        pfizerElement = driver.find_element_by_xpath("/html//div[@id='split-screen-content']/form/div[@class='search-form__fields-row']//label[.='Pfizer-BioNTech COVID Vaccine']")
        pfizerElement.click()
        time.sleep(2)
        
        #enter zip code
        zipElement = driver.find_element_by_xpath(".//*[@id='zipCode']")
        zipElement.send_keys('92646')        
        time.sleep(3)        
        
        #submit zipCode button
        searchElement = driver.find_element_by_xpath("//div[@id='split-screen-content']/form//button[@type='submit']")
        searchElement.click()        
        time.sleep(3)
        
        #find and print vaccine data
        listElement = driver.find_element_by_xpath("//div[@id='split-screen-content']/main/div[3]")        
        print(listElement.text)
        print(type(listElement.text))          
        driver.close()           
        
ff = FindByIdName()
ff.test()

这是我正在寻找的最终输出的示例：

#        Location                            Address                        Distance   Inventory
1.  Vons Pharmacy #3160        8891 Atlanta Ave Huntington Beach, CA 92646   .61 miles   Out Of Stock
2.  Walmart Inc #10-5601       21132 Beach Blvd Huntington Beach, CA 92648   1.05 miles  Out Of Stock
3.  CVS Pharmacy, Inc #09483   10011 Adams Ave  Huntington Beach, CA 92646   1.92 miles  Out Of Stock
...         ...                                 ...                              ...           ...

我在网上看到了一些示例，它们将字符串数据转换为数据帧，但数据看起来是 csv 格式：How to convert a string to a dataframe in Python

任何帮助将不胜感激。我会从你的建议中学到很多东西。谢谢！ =)

Answer 1

如果您保存/返回 result 而不仅仅是打印：

        ...
        result = listElement.text
        driver.close()
        
        return result

然后您可以将split() 和reshape() 放入一个数据框：

result = ff.test().split('\n')

columns = ['number', 'pharmacy', 'address', 'distance', 'status', 'view']
data = np.array(result).reshape(len(result)//len(columns), len(columns))

df = pd.DataFrame(data=data, columns=columns).drop(columns=['number', 'view'])

	pharmacy	address	distance	status
0	VONS PHARMACY #3160	8891 Atlanta Ave • Huntington Beach, CA 92646	0.61 miles	Out Of Stock
1	Walmart Inc #10-5601	21132 Beach Blvd • Huntington Beach, CA 92648	1.05 miles	Out Of Stock
...	...	...	...	...
48	Costco Wholesale Corporation #1110	7562 Center Ave • Huntington Beach, CA 92647	5.96 miles	Out Of Stock
49	Aloha Pharmacy #0596962	15611 Brookhurst St • Westminster, CA 92683	5.99 miles	Out Of Stock