如何将数据存储在CSV文件中，但数据在for循环python中

Question

现在我正在使用 python selenium 进行网页抓取。我想废弃的data in the web browser。 xpath 应该看起来像这样'//*[@id="mainForm:j_idt130_data"]/tr[1]/td[4]'。在我看来，我将tr[1] 更改为tr[str(x+1)]，这样我就可以使用 for 循环打印每一行数据。 td[4] 因为每个数据都在列号4 上。这里是Output by using for loop。

这里是代码：

for x in range(tableLength+1):
    text1 = '//*[@id="mainForm:j_idt130_data"]/tr['
    text2 = ']/td[4]'
    combineText = text1+str(x+1)+text2
    trx = driver.find_element_by_xpath(combineText).text

问题是我不知道如何将所有数据存储到 CSV 文件中。我的数据是数组还是字符串。我试试这段代码：

with open('data.csv', mode='w', newline='') as csv_file:
  csv_writer = csv.writer(csv_file)
  for row in trx:
    csv_writer.writerow(row)

存储在 CSV 文件中的数据是59（这是最后一个数据）。我希望它存储我一直在报废的所有数据。

Answer 1

之前我将 csv 放在 for 循环中，我只是更改了将 for 循环放在 csv 中的结构。比如下面的代码：

with open("data.csv", "w") as csv: 
    # directly go to Summary of Daily Transaction by Agency
    driver.get("http:<link of the current page I want to scrap>")

    # to find first date and last date of the month
    currentDate = datetime.datetime.now()
    firstDate = (currentDate.replace(day=1)).strftime("%d/%m/%Y")
    
    today = datetime.datetime(currentDate.year, currentDate.month, currentDate.day)
    nxt_mnth = today.replace(day=28) + datetime.timedelta(days=4)
    lastDate = (nxt_mnth - datetime.timedelta(days=nxt_mnth.day)).strftime("%d/%m/%Y")
    # 

    # input find first date and last date of the month
    fromDate = driver.find_element_by_name("mainForm:dateFrom_input")
    webdriver.ActionChains(driver).click(fromDate).perform()
    fromDate.send_keys(str(firstDate))
    toDate = driver.find_element_by_name("mainForm:dateTo_input")
    webdriver.ActionChains(driver).click(toDate).perform()
    toDate.send_keys(str(lastDate))
    # 

    agencyList = ['mainForm:agency_2','mainForm:agency_12','mainForm:agency_13','mainForm:agency_14','mainForm:agency_16','mainForm:agency_17','mainForm:agency_18','mainForm:agency_19','mainForm:agency_35','mainForm:agency_36','mainForm:agency_37']
    for K in agencyList:
        dropdown = driver.find_element_by_id("mainForm:agency_label")
        webdriver.ActionChains(driver).click(dropdown).perform()

        agencyName = (driver.find_element_by_id(K).text).replace(',', ';')

        selectAgency = driver.find_element_by_id(K)
        webdriver.ActionChains(driver).click(selectAgency).perform()

        searchBtn = driver.find_element_by_id("mainForm:j_idt128")
        webdriver.ActionChains(driver).click(searchBtn).perform()
        # code end here

        # page usually output 10 data I want to output all 30 data
        selectDropdown = driver.find_element_by_id("mainForm:j_idt130_rppDD")
        all_options = selectDropdown.find_elements_by_tag_name("option")
        for option in all_options:
            option.click()
        #
        sleep(2)
        # to find length of rows table data
        rows = driver.find_elements_by_xpath('//*[@id="mainForm:j_idt130_data"]/tr')
        tableLength = (len(rows))
        #
        
        csv.write(agencyName+ "\n")
    
        for x in range(tableLength):
            
            trx1 = '//*[@id="mainForm:j_idt130_data"]/tr['
            trxTd = ']/td[4]'
            date1 = '//*[@id="mainForm:j_idt130_data"]/tr['
            dateTd = ']/td[2]'
            agencyTd = ']/td[3]'
            agency1 = '//*[@id="mainForm:j_idt130_data"]/tr['
            combineTrx = trx1+str(x+1)+trxTd
            combineDate = date1+str(x+1)+dateTd
            combineAgency = agency1+str(x+1)+agencyTd
            try:
                trx = (driver.find_element_by_xpath(combineTrx).text).replace(',', '') 
                date = (driver.find_element_by_xpath(combineDate).text)
                agency = (driver.find_element_by_xpath(combineAgency).text)

                print(trx)
                csv.write(date+","+agency+","+trx + "\n")
            except NoSuchElementException:
                break #if there no data it will continue go to next agency
        csv.write("\n")

而不是这个结构

for (){
    with open () as csv:
         csv.write
}

您应该这样做以便使用 for 循环将每个数据存储到 CSV 文件中

with open () as csv:
      for (){
          csv.write
       }

Answer 2

这是一个从列表生成 csv 文件的脚本， 假设每个文件的行都是该列表的一个列表元素。

例子：

rows = [['a','b','c'], ['x','y','z']]

>>> csv = ''
>>> for row in rows:
        x = ','.join(map(str,(row)))
        csv += x + '\n'

>>> print(csv)

输出：

a,b,c
x,y,z

然后，您只需将此输出写入扩展名为“.csv”的文件。

您可以将其作为响应或任何需要的内容返回。

例子：

# Return the csv file as an attachment.
return Response(
                 csv_file,
                 mimetype="text/csv",
                 headers={
                        "Content-disposition":"attachment;\
                        filename=my_file.csv"\
                 }
               )