导入带有 url 的 .csv 并处理它们 (PYTHON)

Question

我正在编写一个脚本，它将导入一个 url 列表，然后检查源代码中的一些内容。我需要有关导入 .csv 和处理它的帮助，如果有人可以帮助这里是代码的一部分

from lxml import html
import csv

def main():
with open('urls.csv', 'r') as csvfile:
    urls = [row[0] for row in csv.reader(csvfile)]

for url in urls:

    doc = html.parse(url)
linkziel = 'http://dandydiary.de/de'
if doc.xpath('//a[@href=$url]', url=linkziel):
    for anchor_node in doc.xpath('//a[@href=$url]', url=linkziel):
        if anchor_node.xpath('./ancestor::div[contains(@class, "sidebar")]'):
            print 'Sidebar'
        elif anchor_node.xpath('./parent::div[contains(@class, "widget")]'):
            print 'Sidebar'           
        elif anchor_node.xpath('./ancestor::div[contains(@class, "comment")]'):
            print 'Kommentar'
        elif anchor_node.xpath('./ancestor::div[contains(@id, "comment")]'):
            print 'Kommentar'
        elif anchor_node.xpath('./ancestor::div[contains(@class, "foot")]'):
            print "Footer"
        elif anchor_node.xpath('./ancestor::div[contains(@id, "foot")]'):
            print "Footer" 
        elif anchor_node.xpath('./ancestor::div[contains(@class, "post")]'):
            print "Contextual"         
        else:
            print 'Unidentified Link'          
else:
    print 'Link is Dead'

if __name__ == '__main__':
main()

我不想只指定一个 url，而是想使用一个 csv 来运行（我使用的是 Python 2）

Answer 1

假设您有一个 input.csv 文件，每个新行都有一个 url：

http://de.wikipedia.org
http://spiegel.de
http://www.vickysmodeblog.com/

然后，您可以通过csv 模块将其读入列表并对其进行迭代：

import csv
from lxml import html


with open('input.csv', 'r') as csvfile:
    urls = [row[0] for row in csv.reader(csvfile)]

for url in urls:
    print url

    doc = html.parse(url)
    linkziel = 'http://dandydiary.de/de'
    if doc.xpath('//a[@href=$url]', url=linkziel):
        for anchor_node in doc.xpath('//a[@href=$url]', url=linkziel):
            if anchor_node.xpath('./ancestor::div[contains(@class, "sidebar")]'):
                print 'Sidebar'
            elif anchor_node.xpath('./parent::div[contains(@class, "widget")]'):
                print 'Sidebar'
            elif anchor_node.xpath('./ancestor::div[contains(@class, "comment")]'):
                print 'Kommentar'
            elif anchor_node.xpath('./ancestor::div[contains(@id, "comment")]'):
                print 'Kommentar'
            elif anchor_node.xpath('./ancestor::div[contains(@class, "foot")]'):
                print "Footer"
            elif anchor_node.xpath('./ancestor::div[contains(@id, "foot")]'):
                print "Footer"
            elif anchor_node.xpath('./ancestor::div[contains(@class, "post")]'):
                print "Contextual"
            else:
                print 'Unidentified Link'
    else:
        print 'Link is Dead'

它的输出是：

http://de.wikipedia.org
Link is Dead
http://spiegel.de
Link is Dead
http://www.vickysmodeblog.com/
Contextual

Answer 2

Python 提供了一个csv 模块，您可以使用它来导入列表。