T-SQL - XML 到 SQL Server 表

Question

案例：一个表有一个包含一些 XML 代码的字段。

-- Some XML
'<DTS:ConnectionManager DTS:refId="Package.ConnectionManagers[MTS]" DTS:CreationName="FLATFILE" DTS:DTSID="{296732CC-7D91-4E49-ACD4-384E03BC032E}" DTS:ObjectName="MTS">
    <DTS:PropertyExpression DTS:Name="ConnectionString">@Something</DTS:PropertyExpression>
    <DTS:ObjectData>
        <DTS:ConnectionManager DTS:Format="Delimited" DTS:LocaleID="1033" DTS:HeaderRowDelimiter="_x000D__x000A_" DTS:ColumnNamesInFirstDataRow="True" DTS:RowDelimiter="" DTS:TextQualifier="_x0022_" DTS:CodePage="1252" DTS:ConnectionString="C:\Folder\\File.csv">
            <DTS:FlatFileColumns>
                <DTS:FlatFileColumn DTS:ColumnType="Delimited" DTS:ColumnDelimiter="_x002C_" DTS:MaximumWidth="50" DTS:DataType="129" DTS:TextQualified="True" DTS:ObjectName="MC" DTS:DTSID="{E87E7707-B7F7-4EC6-A2CB-98AD637A3985}" DTS:CreationName="" />
                <DTS:FlatFileColumn DTS:ColumnType="Delimited" DTS:ColumnDelimiter="_x002C_" DTS:DataType="6" DTS:TextQualified="True" DTS:ObjectName="PP" DTS:DTSID="{C7B97962-3B43-40C5-82B1-F6136906CD84}" DTS:CreationName="" />
            </DTS:FlatFileColumns>
        </DTS:ConnectionManager>
    </DTS:ObjectData>
</DTS:ConnectionManager>'
-- Some more XML

想提取一些信息并将其存储为表格格式。

期望的输出

CreationName    ObjectName  ConnectionString        MaximumWidth    DataType    FieldName
FLATFILE        MTS         C:\Folder\\File.csv     50              129         MC
FLATFILE        MTS         C:\Folder\\File.csv     NULL            6           PP

输入输出连接说明

CreationName - DTS:CreationName from DTS:ConnectionManager. i.e. FLATFILE
ObjectName - DTS:ObjectName from DTS:ConnectionManager. i.e. MTS
ConnectionString - DTS:ConnectionString from DTS:ObjectData\DTS:ConnectionManager. i.e. "C:\Folder\\File.csv"
MaximumWidth - DTS:MaximumWidth from DTS:FlatFileColumns i.e. 50 -- NOTE: MaximumWidth might not always exist
DataType - DTS:DataType from DTS:FlatFileColumns i.e. 129
FieldName - DTS:ObjectName from DTS:FlatFileColumns i.e. MC

对 SQL Server 中的 XML 没有太多经验。 （如果我找到有意义的地方，我会做一些自己的游戏并在这里发布。:)）

更新的 XML 示例

    <DTS:Executable xmlns:DTS="www.microsoft.com/SqlServer/Dts" DTS:refId="P" DTS:CreationDate="10/01/2015 12:00:00">
  <DTS:ConnectionManagers>
    <DTS:ConnectionManager DTS:refId="Package.ConnectionManagers[FF]" DTS:CreationName="FLATFILE" DTS:DTSID="{123}" DTS:ObjectName="FF">
      <DTS:ObjectData>
        <DTS:ConnectionManager DTS:Format="Delimited" DTS:LocaleID="1033" DTS:HeaderRowDelimiter="_x000D__x000A_" DTS:ColumnNamesInFirstDataRow="True" DTS:RowDelimiter="" DTS:TextQualifier="_x0022_" DTS:CodePage="1252" DTS:ConnectionString="Test.csv">
          <DTS:FlatFileColumns>
            <DTS:FlatFileColumn DTS:ColumnType="Delimited" DTS:ColumnDelimiter="_x002C_" DTS:DataType="11" DTS:TextQualified="True" DTS:ObjectName="TestCN" DTS:DTSID="{012}" DTS:CreationName="" />
          </DTS:FlatFileColumns>
        </DTS:ConnectionManager>
      </DTS:ObjectData>
    </DTS:ConnectionManager>
    <DTS:ConnectionManager DTS:refId="Package.ConnectionManagers[FF2]" DTS:CreationName="FLATFILE" DTS:DTSID="{123}" DTS:ObjectName="FF2">
      <DTS:ObjectData>
        <DTS:ConnectionManager DTS:Format="Delimited" DTS:LocaleID="1033" DTS:HeaderRowDelimiter="_x000D__x000A_" DTS:ColumnNamesInFirstDataRow="True" DTS:RowDelimiter="" DTS:TextQualifier="_x0022_" DTS:CodePage="1252" DTS:ConnectionString="Test2.csv">
          <DTS:FlatFileColumns>
            <DTS:FlatFileColumn DTS:ColumnType="Delimited" DTS:ColumnDelimiter="_x002C_" DTS:DataType="11" DTS:TextQualified="True" DTS:ObjectName="TestCN2" DTS:DTSID="{012}" DTS:CreationName="" />
          </DTS:FlatFileColumns>
        </DTS:ConnectionManager>
      </DTS:ObjectData>
    </DTS:ConnectionManager>
  </DTS:ConnectionManagers>
</DTS:Executable>

Answer 1

当然，您也可以使用通用语言将 XML 内容转换为表格格式，以进行数据库导入或分隔文件导出。

尽管它很棒，但 SQL 是一种专用语言，不像 Java、C#、Python、PHP、Perl、VB 和其他还带有运行 XPath、XSLT 和其他 XML 的库的语言那样流畅或动态- 具体任务。此外，这些语言可以连接到任何数据库以检索 BLOB 数据。

对于未来的读者，以下是使用 OP 数据需求的开源示例。您会注意到在 xpaths 中使用带有 [] 的方括号，允许更多 DTS:ConnectionManager 元素：

Python（使用 lxml 库）

import os
import lxml.etree as ET

cd = os.path.dirname(os.path.abspath(__file__))

xmlfile = 'DTSfile.xml'
dom = ET.parse(os.path.join(cd, xmlfile))
root = dom.getroot()

nodexpath = dom.xpath("//DTS:ConnectionManager", namespaces=root.nsmap)
dataline = []

def checkPath(xpathstr):    
    if dom.xpath(xpathstr, namespaces=root.nsmap) == []:
        return ''
    else:
        return dom.xpath(xpathstr, namespaces=root.nsmap)[0]

for i in range(1,len(nodexpath)+1):
    if i % 2 == 0: continue

    dataline = []    
    dataline.append(checkPath('//DTS:ConnectionManager[{0}]/@DTS:CreationName'.format(i)))
    dataline.append(checkPath('//DTS:ConnectionManager[{0}]/@DTS:ObjectName'.format(i)))
    dataline.append(checkPath('//DTS:ConnectionManager[{0}]/DTS:ObjectData/DTS:ConnectionManager/@DTS:ConnectionString'.format(i)))    
    dataline.append(checkPath('//DTS:ConnectionManager[{0}]/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn/@DTS:MaximumWidth'.format(i)))
    dataline.append(checkPath('//DTS:ConnectionManager[{0}]/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn/@DTS:DataType'.format(i)))
    dataline.append(checkPath('//DTS:ConnectionManager[{0}]/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn/@DTS:ObjectName'.format(i)))

    print(dataline)

['FLATFILE', 'FF', 'Test.csv', '', '11', 'TestCN']
['FLATFILE', 'FF2', 'Test2.csv', '', '11', 'TestCN2']

PHP（使用 simple_xml 对象）

$cd = dirname(__FILE__);

$xml = simplexml_load_file($cd.'/DTSfile.xml');
$xml->registerXPathNamespace('DTS', 'www.microsoft.com/SqlServer/Dts');

$values = [];
$node = $xml->xpath('//DTS:ConnectionManager');

function checkPath($xml, $xpathstr) {         
     $path = $xml->xpath($xpathstr);
     foreach ($path as $value) {
      if (count($path) > 0) {
           foreach($path as $value) {           
           return $value;         
           }
      }   
      else {
           return '';
      }       
     }         
}
$i = 1;

foreach ($node as $n){         
     $values[] = checkPath($xml, '//DTS:ConnectionManager['.$i.']/@DTS:CreationName');
     $values[] = checkPath($xml, '//DTS:ConnectionManager['.$i.']/@DTS:ObjectName');
     $values[] = checkPath($xml, '//DTS:ConnectionManager['.$i.']/DTS:ObjectData/DTS:ConnectionManager/@DTS:ConnectionString');    
     $values[] = checkPath($xml, '//DTS:ConnectionManager['.$i.']/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn/@DTS:MaximumWidth');
     $values[] = checkPath($xml, '//DTS:ConnectionManager['.$i.']/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn/@DTS:DataType');
     $values[] = checkPath($xml, '//DTS:ConnectionManager['.$i.']/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn/@DTS:ObjectName');     

     if ($values[1] != "") {
      echo implode(", ", $values)."\n";
     }
     $i++;
     $values = [];         
}

FLATFILE, FF, Test.csv, , 11, TestCN
FLATFILE, FF2, Test2.csv, , 11, TestCN2

R（使用xml库）

library(XML) 

setwd("C:\\Path\\To\\Working\\Directory")

doc<-xmlParse("DTSfile.xml")

nodes <- as.list(xpathSApply(doc, '//DTS:ConnectionManager'))

checkPath <- function (xpathstr) {
  if (length(as.list(xpathSApply(doc, xpathstr))) > 0) {
    return(as.list(xpathSApply(doc, xpathstr)))
  } else {
    return("")
  }
}

for (i in (1:length(nodes))) {  
  if (i %% 2) {
    data <- checkPath(sprintf('//DTS:ConnectionManager[%d]/@DTS:CreationName', i))
    data <- append(data, checkPath(sprintf('//DTS:ConnectionManager[%d]/@DTS:ObjectName', i)))
    data <- append(data, checkPath(sprintf('//DTS:ConnectionManager[%d]/DTS:ObjectData/DTS:ConnectionManager/@DTS:ConnectionString', i)))
    data <- append(data, checkPath(sprintf('//DTS:ConnectionManager[%d]/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn/@DTS:MaximumWidth', i)))
    data <- append(data, checkPath(sprintf('//DTS:ConnectionManager[%d]/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn/@DTS:DataType', i)))
    data <- append(data, checkPath(sprintf('//DTS:ConnectionManager[%d]/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn/@DTS:ObjectName', i)))

    print(paste(data, collapse = ', '))
  }

}

[1] "FLATFILE, FF, Test.csv, , 11, TestCN"
[1] "FLATFILE, FF2, Test2.csv, , 11, TestCN2"

Answer 2

你没有在你的根元素中声明你的命名空间，所以我替换了它。尽管我是在 2014 年编写的，但它应该是自解压的，并且可以在我猜测的 2008 年及更高版本中运行。只需将其弹出到 SQL Server Management Studio 中即可：

太平洋标准时间下午 1:45 更新：

感谢 Shnugo 简化了“With XMLNamespaces”。

DECLARE @XML XML = '
<DTS:Executable xmlns:DTS="www.microsoft.com/SqlServer/Dts" DTS:refId="P" DTS:CreationDate="10/01/2015 12:00:00">
  <DTS:ConnectionManagers>
    <DTS:ConnectionManager DTS:refId="Package.ConnectionManagers[FF]" DTS:CreationName="FLATFILE" DTS:DTSID="{123}" DTS:ObjectName="FF">
      <DTS:ObjectData>
        <DTS:ConnectionManager DTS:Format="Delimited" DTS:LocaleID="1033" DTS:HeaderRowDelimiter="_x000D__x000A_" DTS:ColumnNamesInFirstDataRow="True" DTS:RowDelimiter="" DTS:TextQualifier="_x0022_" DTS:CodePage="1252" DTS:ConnectionString="Test.csv">
          <DTS:FlatFileColumns>
            <DTS:FlatFileColumn DTS:ColumnType="Delimited" DTS:ColumnDelimiter="_x002C_" DTS:DataType="11" DTS:TextQualified="True" DTS:ObjectName="TestCN" DTS:DTSID="{012}" DTS:CreationName="" />
          </DTS:FlatFileColumns>
        </DTS:ConnectionManager>
      </DTS:ObjectData>
    </DTS:ConnectionManager>
  </DTS:ConnectionManagers>
</DTS:Executable>'
;

WITH XMLNAMESPACES (N'www.microsoft.com/SqlServer/Dts' as DTS )
SELECT 
    y.vals.query('.') AS NodesAsExtracted
,   x.vals.value('@DTS:CreationName', 'Varchar(255)') AS CreationName
,   x.vals.value('@DTS:ObjectName', 'Varchar(255)') AS ObjectName
,   y.vals.value('@DTS:ConnectionString', 'Varchar(255)') AS ConnectionString
,   x.vals.value('@DTS:ColumnType', 'Varchar(255)') AS ColumnType
,   x.vals.value('@DTS:MaximumWidth', 'Varchar(255)') AS MaximumWidth
FROM @XML.nodes('/DTS:Executable/DTS:ConnectionManagers/DTS:ConnectionManager/DTS:ObjectData/DTS:ConnectionManager') AS y(vals)
    CROSS APPLY @XML.nodes('/DTS:Executable/DTS:ConnectionManagers/DTS:ConnectionManager/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn') AS x(vals)


/*
The key piece is you are extracting data with a namespace, which makes things harder when querying.
You need to repeat certain 'nodes' so there is a syntax for that called originally enough 'nodes' that breaks up a 3d object like xml into multiple bits
I do one for the high level and one for the lower and then cross apply them which really is a whole world into itself I won't mention here
It should be represented as a parent 'x' and the values found 'vals'
I showed an example as is first when I query '('.')' which is everything in essence.
My namespace declaration must match on the xml that exists and the declaration.

more on nodes https://msdn.microsoft.com/en-us/library/ms188282.aspx
more on query https://msdn.microsoft.com/en-us/library/ms191474.aspx
more on value https://msdn.microsoft.com/en-us/library/ms178030.aspx
*/

Answer 3

这是对 djangojazz 答案的增强。不要接受这个，它只是一个副本（但你可以投票，如果你喜欢它;-) ...

通过使用 WITH XMLNAMESPACES 可以避免命名空间的多重声明：

WITH XMLNAMESPACES (N'http://DTS' as DTS )
SELECT 
    x.vals.query('.') AS NodesAsExtracted
,   x.vals.value('@DTS:CreationName', 'Varchar(255)') AS CreationName
,   x.vals.value('@DTS:ObjectName', 'Varchar(255)') AS ObjectName
,   y.vals.value('@DTS:ConnectionString', 'Varchar(255)') AS ConnectionString
,   x.vals.value('@DTS:ColumnType', 'Varchar(255)') AS ColumnType
,   x.vals.value('@DTS:MaximumWidth', 'Varchar(255)') AS MaximumWidth
from @XML.nodes('/DTS:ConnectionManager/DTS:ObjectData/DTS:ConnectionManager/DTS:FlatFileColumns/DTS:FlatFileColumn') AS x(vals)
    CROSS APPLY @XML.nodes('/DTS:ConnectionManager/DTS:ObjectData/DTS:ConnectionManager') AS y(vals)