【问题标题】:How to iterate over XML tags in Python using ElementTree & save to CSV如何使用 ElementTree 在 Python 中迭代 XML 标记并保存到 CSV
【发布时间】:2019-08-21 10:32:12
【问题描述】:

我正在尝试使用 ElementTree 遍历树中的所有节点和子节点。我想将所有父及其子 XML 标记作为列和值,以 CSV 格式将子节点附加到父节点。我正在使用python 2.7。标题应该只打印一次&下面应该是各自的值

XML 文件:

<Customers>  
<Customer CustomerID="GREAL">  
      <CompanyName>Great Lakes Food Market</CompanyName>  
      <ContactName>Howard Snyder</ContactName>  
      <ContactTitle>Marketing Manager</ContactTitle>  
      <Phone>(503) 555-7555</Phone>  
      <FullAddress>  
        <Address>2732 Baker Blvd.</Address>  
        <City>Eugene</City>  
        <Region>OR</Region>  
        <PostalCode>97403</PostalCode>  
        <Country>USA</Country>  
      </FullAddress>  
 </Customer>  
    <Customer CustomerID="HUNGC">  
      <CompanyName>Hungry Coyote Import Store</CompanyName>  
      <ContactName>Yoshi Latimer</ContactName>  
      <ContactTitle>Sales Representative</ContactTitle>  
      <Phone>(503) 555-6874</Phone>  
      <Fax>(503) 555-2376</Fax>  
      <FullAddress>  
        <Address>City Center Plaza 516 Main St.</Address>  
        <City>Elgin</City>  
        <Region>OR</Region>  
        <PostalCode>97827</PostalCode>  
        <Country>USA</Country>  
      </FullAddress>  
    </Customer>  
    <Customer CustomerID="LAZYK">  
      <CompanyName>Lazy K Kountry Store</CompanyName>  
      <ContactName>John Steel</ContactName>  
      <ContactTitle>Marketing Manager</ContactTitle>  
      <Phone>(509) 555-7969</Phone>  
      <Fax>(509) 555-6221</Fax>  
      <FullAddress>  
        <Address>12 Orchestra Terrace</Address>  
        <City>Walla Walla</City>  
        <Region>WA</Region>  
        <PostalCode>99362</PostalCode>  
        <Country>USA</Country>  
      </FullAddress>  
    </Customer>  
    <Customer CustomerID="LETSS">  
      <CompanyName>Let's Stop N Shop</CompanyName>  
      <ContactName>Jaime Yorres</ContactName>  
      <ContactTitle>Owner</ContactTitle>  
      <Phone>(415) 555-5938</Phone>  
      <FullAddress>  
        <Address>87 Polk St. Suite 5</Address>  
        <City>San Francisco</City>  
        <Region>CA</Region>  
        <PostalCode>94117</PostalCode>  
        <Country>USA</Country>  
      </FullAddress>  
    </Customer>  
  </Customers>  

我的代码:

#Import Libraries
import csv
import xmlschema
import xml.etree.ElementTree as ET

#Define the variable to store the XML Document
xml_file = 'C:/Users/391648/Desktop/BOSS_20190618_20190516_18062019141928_CUMA/source_Files_XML/CustomersOrders.xml'

#using XML Schema Library validate the XML against XSD
my_schema = xmlschema.XMLSchema('C:/Users/391648/Desktop/BOSS_20190618_20190516_18062019141928_CUMA/source_Files_XML/CustomersOrders.xsd')
SchemaCheck = my_schema.is_valid(xml_file)
print(SchemaCheck) #Prints as True if the document is validated with XSD

#Parse XML & get root
tree = ET.parse(xml_file)
root = tree.getroot()

#Create & Open CSV file
xml_data_to_csv = open('C:/Users/391648/Desktop/BOSS_20190618_20190516_18062019141928_CUMA/source_Files_XML/PythonXMl.csv','w')

#create variable to write to csv
csvWriter = csv.writer(xml_data_to_csv)

#Create list contains header
count =0

#Loop for each node
for element in root.findall('Customers/Customer'):
    List_nodes = []

    #Get head by Tag
    if count ==0:
        list_header =[]
        Full_Address = []
        CompanyName = element.find('CompanyName').tag
        list_header.append(CompanyName)

        ContactName = element.find('ContactName').tag
        list_header.append(ContactName)

        ContactTitle = element.find('ContactTitle').tag
        list_header.append(ContactTitle)

        Phone = element.find('Phone').tag
        list_header.append(Phone)

        print(list_header)
        csvWriter.writerow(list_header)

        count = count + 1

    #Get the data of the Node
    CompanyName = element.find('CompanyName').text
    List_nodes.append(CompanyName)

    ContactName = element.find('ContactName').text
    List_nodes.append(ContactName)

    ContactTitle = element.find('ContactTitle').text
    List_nodes.append(ContactTitle)

    Phone = element.find('Phone').text
    List_nodes.append(Phone)

    print(List_nodes)

    #Write List_Nodes to CSV
    csvWriter.writerow(List_nodes)

xml_data_to_csv.close()
Expected CSV output:

CompanyName,ContactName,ContactTitle,Phone, Address, City, Region, PostalCode, Country
Great Lakes Food Market,Howard Snyder,Marketing Manager,(503) 555-7555, City Center Plaza 516 Main St., Elgin, OR, 97827, USA
Hungry Coyote Import Store,Yoshi Latimer,Sales Representative,(503) 555-6874, 12 Orchestra Terrace, Walla Walla, WA, 99362, USA

【问题讨论】:

  • 不应该是root.findall('Customer')吗?

标签: python xml csv nodes


【解决方案1】:

使用lxml 可能会更好。它具有查找内置元素所需的大部分功能。

from lxml import etree
import csv

with open('file.xml') as fp:
    xml = etree.fromstring(fp.read())

field_dict = {
    'CompanyName': 'CompanyName',
    'ContactName': 'ContactName',
    'ContactTitle': 'ContactTitle',
    'Phone': 'Phone',
    'Address': 'FullAddress/Address',
    'City': 'FullAddress/City',
    'Region': 'FullAddress/Region',
    'PostalCode': 'FullAddress/PostalCode',
    'Country': 'FullAddress/Country'
}

customers = []
for customer in xml:
    line = {k: customer.find(v).text for k, v in field_dict.items()}
    customers.append(line)

with open('customers.csv', 'w') as fp:
    writer = csv.DictWriter(fp, field_dict)
    writer.writerows(customers)

【讨论】:

    【解决方案2】:

    您可以使用xmltodict将数据转换为JSON格式,而不是解析XML:

    import xmltodict
    import pandas as pd
    
    with open('data.xml', 'r') as f:
        data = xmltodict.parse(f.read())['Customers']['Customer']
    
    data_pd = {'CompanyName': [i['CompanyName'] for i in data],
               'ContactName': [i['ContactName'] for i in data],
               'ContactTitle': [i['ContactTitle'] for i in data],
               'Phone': [i['Phone'] for i in data],
               'Address': [i['FullAddress']['Address'] for i in data],
               'City': [i['FullAddress']['City'] for i in data],
               'Region': [i['FullAddress']['Region'] for i in data],
               'PostalCode': [i['FullAddress']['PostalCode'] for i in data],
               'Country': [i['FullAddress']['Country'] for i in data]}
    
    df = pd.DataFrame(data_pd)
    df.to_csv('result.csv', index=False)
    

    输出 CSV 文件:

    CompanyName,ContactName,ContactTitle,Phone,Address,City,Region,PostalCode,Country
    Great Lakes Food Market,Howard Snyder,Marketing Manager,(503) 555-7555,2732 Baker Blvd.,Eugene,OR,97403,USA
    Hungry Coyote Import Store,Yoshi Latimer,Sales Representative,(503) 555-6874,City Center Plaza 516 Main St.,Elgin,OR,97827,USA
    Lazy K Kountry Store,John Steel,Marketing Manager,(509) 555-7969,12 Orchestra Terrace,Walla Walla,WA,99362,USA
    Let's Stop N Shop,Jaime Yorres,Owner,(415) 555-5938,87 Polk St. Suite 5,San Francisco,CA,94117,USA
    

    【讨论】:

    • 嗨,我在运行您的代码 sn-p 时遇到以下错误:Traceback(最近一次调用最后一次):文件“C:/Users/391648/PycharmProjects/XSD_XMl_Parse/Parse_XML_Df_Pandas.py”,第 5 行,在 data = xmltodict.parse(f.read())['Customers']['Customer'] KeyError: 'Customers' Process finished with exit code 1. 这可能是什么原因?
    • 您是否尝试过使用与您的问题相同的 XML?
    【解决方案3】:

    我改变了几件事:

    • 删除了架构验证,因为我没有 XSD。您可以包含它
    • 使子节点遍历动态而不是静态引用每个子节点
    • 主要的for loop 条件从for customer in root.findall('Customers/Customer') 更改为for customer in root.findall('Customer')

    但是,我试图保持您的程序结构和库使用不变。这是修改后的程序:

    import xml.etree.ElementTree as et
    import csv
    
    tree = et.parse("../data/customers.xml")
    root = tree.getroot()
    headers = []
    count = 0
    xml_data_to_csv = open('../data/customers.csv', 'w')
    
    csvWriter = csv.writer(xml_data_to_csv)
    for customer in root.findall('Customer'):
        data = []
        for detail in customer:
            if(detail.tag == 'FullAddress'):
                for addresspart in detail:
                    data.append(addresspart.text.rstrip('/n/r'))
                    if(count == 0):
                        headers.append(addresspart.tag)
            else:
                data.append(detail.text.rstrip('/n/r'))
                if(count == 0):
                    headers.append(detail.tag)
        if(count == 0):
            csvWriter.writerow(headers)
        csvWriter.writerow(data)
        count = count + 1
    

    使用给定的输入 XML 它产生的内容:

    CompanyName,ContactName,ContactTitle,Phone,Address,City,Region,PostalCode,Country
    Great Lakes Food Market,Howard Snyde,Marketing Manage,(503) 555-7555,2732 Baker Blvd.,Eugene,OR,97403,USA
    Hungry Coyote Import Store,Yoshi Latime,Sales Representative,(503) 555-6874,(503) 555-2376,City Center Plaza 516 Main St.,Elgi,OR,97827,USA
    Lazy K Kountry Store,John Steel,Marketing Manage,(509) 555-7969,(509) 555-6221,12 Orchestra Terrace,Walla Walla,WA,99362,USA
    Let's Stop N Shop,Jaime Yorres,Owne,(415) 555-5938,87 Polk St. Suite 5,San Francisco,CA,94117,USA
    

    注意:您可以追加到数组并一次性写入,而不是在循环中写入 CSV。这取决于您的内容大小和性能。


    更新:当您在 XML 中有客户及其订单时

    XML 处理和 CSV 编写代码结构保持不变。此外,在处理客户时处理Orders 元素。现在,在OrdersOrder 下的元素可以像Customer 一样被处理。正如您提到的,每个Order 也有ShipInfo

    假设输入的 XML 是(基于下面的评论):

    <Customers>
        <Customer CustomerID="GREAL">
            <CompanyName>Great Lakes Food Market</CompanyName>
            <ContactName>Howard Snyder</ContactName>
            <ContactTitle>Marketing Manager</ContactTitle>
            <Phone>(503) 555-7555</Phone>
            <FullAddress>
                <Address>2732 Baker Blvd.</Address>
                <City>Eugene</City>
                <Region>OR</Region>
                <PostalCode>97403</PostalCode>
                <Country>USA</Country>
            </FullAddress>
            <Orders>
                <Order>
                    <Param1>Value1</Param1>
                    <Param2>Value2</Param2>
                    <ShipInfo>
                        <ShipInfoParam1>Value3</ShipInfoParam1>
                        <ShipInfoParam2>Value4</ShipInfoParam2>
                    </ShipInfo>
                </Order>
                <Order>
                    <Param1>Value5</Param1>
                    <Param2>Value6</Param2>
                    <ShipInfo>
                        <ShipInfoParam1>Value7</ShipInfoParam1>
                        <ShipInfoParam2>Value8</ShipInfoParam2>
                    </ShipInfo>
                </Order>
            </Orders>
        </Customer>
        <Customer CustomerID="HUNGC">
            <CompanyName>Hungry Coyote Import Store</CompanyName>
            <ContactName>Yoshi Latimer</ContactName>
            <ContactTitle>Sales Representative</ContactTitle>
            <Phone>(503) 555-6874</Phone>
            <Fax>(503) 555-2376</Fax>
            <FullAddress>
                <Address>City Center Plaza 516 Main St.</Address>
                <City>Elgin</City>
                <Region>OR</Region>
                <PostalCode>97827</PostalCode>
                <Country>USA</Country>
            </FullAddress>
            <Orders>
                <Order>
                    <Param1>Value7</Param1>
                    <Param2>Value8</Param2>
                    <ShipInfo>
                        <ShipInfoParam1>Value9</ShipInfoParam1>
                        <ShipInfoParam2>Value10</ShipInfoParam2>
                    </ShipInfo>
                </Order>
            </Orders>
        </Customer>
        <Customer CustomerID="LAZYK">
            <CompanyName>Lazy K Kountry Store</CompanyName>
            <ContactName>John Steel</ContactName>
            <ContactTitle>Marketing Manager</ContactTitle>
            <Phone>(509) 555-7969</Phone>
            <Fax>(509) 555-6221</Fax>
            <FullAddress>
                <Address>12 Orchestra Terrace</Address>
                <City>Walla Walla</City>
                <Region>WA</Region>
                <PostalCode>99362</PostalCode>
                <Country>USA</Country>
            </FullAddress>
        </Customer>
        <Customer CustomerID="LETSS">
            <CompanyName>Let's Stop N Shop</CompanyName>
            <ContactName>Jaime Yorres</ContactName>
            <ContactTitle>Owner</ContactTitle>
            <Phone>(415) 555-5938</Phone>
            <FullAddress>
                <Address>87 Polk St. Suite 5</Address>
                <City>San Francisco</City>
                <Region>CA</Region>
                <PostalCode>94117</PostalCode>
                <Country>USA</Country>
            </FullAddress>
        </Customer>
    </Customers>
    

    这是处理客户和订单的修改后的代码:

    import xml.etree.ElementTree as et
    import csv
    
    tree = et.parse("../data/customers-with-orders.xml")
    root = tree.getroot()
    
    customer_csv = open('../data/customers-part.csv', 'w')
    order_csv = open('../data/orders-part.csv', 'w')
    
    customerCsvWriter = csv.writer(customer_csv)
    orderCsvWriter = csv.writer(order_csv)
    
    customerHeaders = []
    orderHeaders = ['CustomerID']
    isFirstCustomer = True
    isFirstOrder = True
    
    
    def processOrders(customerId):
        global isFirstOrder
        for order in detail.findall('Order'):
            orderData = [customerId]
            for orderdetail in order:
                if(orderdetail.tag == 'ShipInfo'):
                    for shipinfopart in orderdetail:
                        orderData.append(shipinfopart.text.rstrip('/n/r'))
                        if(isFirstOrder):
                            orderHeaders.append(shipinfopart.tag)
                else:
                    orderData.append(orderdetail.text.rstrip('/n/r'))
                    if(isFirstOrder):
                        orderHeaders.append(orderdetail.tag)
            if(isFirstOrder):
                orderCsvWriter.writerow(orderHeaders)
            orderCsvWriter.writerow(orderData)
            isFirstOrder = False
    
    
    for customer in root.findall('Customer'):
        customerData = []
        customerId = customer.get('CustomerID')
        for detail in customer:
            if(detail.tag == 'FullAddress'):
                for addresspart in detail:
                    customerData.append(addresspart.text.rstrip('/n/r'))
                    if(isFirstCustomer):
                        customerHeaders.append(addresspart.tag)
            elif(detail.tag == 'Orders'):
                processOrders(customerId)
            else:
                customerData.append(detail.text.rstrip('/n/r'))
                if(isFirstCustomer):
                    customerHeaders.append(detail.tag)
        if(isFirstCustomer):
            customerCsvWriter.writerow(customerHeaders)
        customerCsvWriter.writerow(customerData)
        isFirstCustomer = False
    

    customers-part.csv 中产生的输出:

    CompanyName,ContactName,ContactTitle,Phone,Address,City,Region,PostalCode,Country
    Great Lakes Food Market,Howard Snyde,Marketing Manage,(503) 555-7555,2732 Baker Blvd.,Eugene,OR,97403,USA
    Hungry Coyote Import Store,Yoshi Latime,Sales Representative,(503) 555-6874,(503) 555-2376,City Center Plaza 516 Main St.,Elgi,OR,97827,USA
    Lazy K Kountry Store,John Steel,Marketing Manage,(509) 555-7969,(509) 555-6221,12 Orchestra Terrace,Walla Walla,WA,99362,USA
    Let's Stop N Shop,Jaime Yorres,Owne,(415) 555-5938,87 Polk St. Suite 5,San Francisco,CA,94117,USA
    

    orders-part.csv 中产生的输出:

    CustomerID,Param1,Param2,ShipInfoParam1,ShipInfoParam2
    GREAL,Value1,Value2,Value3,Value4
    GREAL,Value5,Value6,Value7,Value8
    HUNGC,Value7,Value8,Value9,Value10
    

    注意:代码可以通过重用进一步优化。我把那部分留给你。其次,请注意,在每个订单中都添加了客户 ID,以便区分。

    【讨论】:

    • 您好,感谢您提供代码 sn-p。如果必须在单个 CSV 文件或多个 CSV 文件中合并多个元素,那么我应该如何处理?假设我们有 XML 格式的客户和订单数据集。 1. 客户有多个孩子,订单有多个,但我们可以将两个元素数据都包含在一个 CSV 中,彼此相邻或不同的 CSV。
    • 我建议您为客户和订单数据使用两个不同的 CSV 文件。解析逻辑代码结构与上图相同。当您在Customer 下遇到“Order”元素时,请写信给另一个CSV。另外,请注意 Order CSV 标头。
    • 在同一个 XML 文件中,有 Orders 元素,就像 Customers 一样有子元素。例如: 1. 客户 > 客户 > 完整地址 2. 订单 > 订单 > ShipInfo。我试图在相同的代码中添加相同的解析逻辑,但它不起作用。我需要做不同的事情吗?
    • @RameezShaikh 当你说它不起作用时还不清楚。请查看更新后的答案(下一部分),了解如何处理客户和订单,并将输出写入两个单独的文件。
    • 感谢您发布代码 sn-p。我已经根据我的需要修改了代码并且它有效。 :)
    猜你喜欢
    • 1970-01-01
    • 2018-07-04
    • 2023-02-09
    • 2015-04-30
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2012-02-17
    • 1970-01-01
    相关资源
    最近更新 更多