通过 SSIS 仅导入 Excel 的最后一列

Question

我有一个每天都会收到的 excel 文件。该文件中的列数不是特定的。我的要求只是通过 SSIS 加载表中的最后一列。如何动态识别上次使用的列？

Answer 1

你可以使用c#脚本：

确保添加 Using System.Data.OleDb;到命名空间区域 并添加输出列 LastCol 并选择数据类型。

public override void CreateNewOutputRows()
    {
        /*
          Add rows by calling the AddRow method on the member variable named "<Output Name>Buffer".
          For example, call MyOutputBuffer.AddRow() if your output was named "MyOutput".
        */
        string fileName = @"C:\test.xlsx";
        string SheetName = "Sheet1";
        string cstr = "Provider.ACE.OLEDB.12.0;Data Source=" + fileName + ";Extended Properties=\"Excel 12.0;HDR=No;IMEX=1\"";

    OleDbConnection xlConn = new OleDbConnection(cstr);
    xlConn.Open();

    OleDbCommand xlCmd = xlConn.CreateCommand();
    xlCmd.CommandText = "Select * from [" + SheetName + "]";
    xlCmd.CommandType = CommandType.Text;
    OleDbDataReader rdr = xlCmd.ExecuteReader();

    int rowCt = 0; //Counter

    while (rdr.Read())
    {
        //skip headers
        if (rowCt != 0)
        {
            int maxCol = rdr.FieldCount;
            Output0Buffer.AddRow();
            Output0Buffer.LastCol = (int)rdr[maxCol];
        }
        rowCt++; //increment counter
    }
}

Answer 2

解决方案概述

使用脚本任务：

• 获取最后一列索引

• 使用以下函数将索引转换为列字母（例如：1 -> A）

  Private Function GetExcelColumnName(columnNumber As Integer) As String
      Dim dividend As Integer = columnNumber
      Dim columnName As String = String.Empty
      Dim modulo As Integer
  
      While dividend > 0
         modulo = (dividend - 1) Mod 26
         columnName = Convert.ToChar(65 + modulo).ToString() & columnName
         dividend = CInt((dividend - modulo) / 26)
     End While
  
     Return columnName
  End Function
  

• 构建只读最后一列的 SQL 命令

• 选择此查询作为 Excel 源

详细解决方案

这个答案是假设工作表名称是Sheet1，使用的编程语言是VB.Net

1. 首先创建一个字符串类型的 SSIS 变量（即@[User::strQuery]）
2. 添加另一个包含 Excel 文件路径的变量（即@[User::ExcelFilePath]）
3. 添加一个脚本任务，选择@[User::strQuery]作为读写变量，@[User::ExcelFilePath]作为只读变量（在脚本任务窗口中）
4. 将脚本语言设置为 VB.Net，并在脚本编辑器窗口中编写以下脚本：

注意：你必须导入 System.Data.OleDb

    m_strExcelPath = Dts.Variables.Item("ExcelFilePath").Value.ToString

    Dim strSheetname As String = String.Empty
    Dim intLastColumn As Integer = 0

    m_strExcelConnectionString = Me.BuildConnectionString()
    Try


        Using OleDBCon As New OleDbConnection(m_strExcelConnectionString)

            If OleDBCon.State <> ConnectionState.Open Then
                OleDBCon.Open()
            End If

            'Get all WorkSheets
            m_dtschemaTable = OleDBCon.GetOleDbSchemaTable(OleDbSchemaGuid.Tables,
                                                               New Object() {Nothing, Nothing, Nothing, "TABLE"})

            'Loop over work sheet to get the first one (the excel may contains temporary sheets or deleted ones

            For Each schRow As DataRow In m_dtschemaTable.Rows
                strSheetname = schRow("TABLE_NAME").ToString

                If Not strSheetname.EndsWith("_") AndAlso strSheetname.EndsWith("$") Then

                    Using cmd As New OleDbCommand("SELECT * FROM [" & strSheetname & "]", OleDBCon)

                        Dim dtTable As New DataTable("Table1")


                        cmd.CommandType = CommandType.Text

                        Using daGetDataFromSheet As New OleDbDataAdapter(cmd)

                            daGetDataFromSheet.Fill(dtTable)

                        End Using
                    'Get the last Column Index
                    intLastColumn =  dtTable.Columns.Count

                    End Using

                    'when the first correct sheet is found there is no need to check others
                    Exit For

                End If
            Next

            OleDBCon.Close()

        End Using

    Catch ex As Exception
        Throw New Exception(ex.Message, ex)
    End Try

    Dim strColumnname as String = GetExcelColumnName(intLastColumn)
    Dts.Variables.Item("strQuery").Value = "SELECT * FROM [" & strSheetname & strColumnname & ":" & strColumnname & "]"

    Dts.TaskResult = ScriptResults.Success
End Sub


Private Function GetExcelColumnName(columnNumber As Integer) As String
    Dim dividend As Integer = columnNumber
    Dim columnName As String = String.Empty
    Dim modulo As Integer

    While dividend > 0
       modulo = (dividend - 1) Mod 26
       columnName = Convert.ToChar(65 + modulo).ToString() & columnName
       dividend = CInt((dividend - modulo) / 26)
   End While

   Return columnName
End Function

5. 然后你必须添加一个 Excel 连接管理器，并选择你要导入的 excel 文件（只需选择一个示例来定义元数据）
6. 将默认值Select * from [Sheet1$] 分配给变量@[User::strQuery]
7. 在数据流任务中添加一个 Excel 源，从变量中选择 SQL 命令，然后选择@[User::strQuery]
8. 将 DataFlow 任务 Delay Validation 属性设置为 True
9. 将其他组件添加到 DataFlow 任务中

---

参考文献

• Importing excel files having variable headers
• Converting Numbers to Excel Letter Column vb.net

Answer 3

不，你不能那样做。列数和数据类型必须事先确定，不能更改。否则 SSIS 将失败。所以没有办法动态获取最后一列。解决方法可能是使用某个宏从 excel 内部获取最后一列，然后将其用作 SSIS 的源。