在 SSIS 中使用脚本任务导入 Json 文件答案

【问题标题】：Importing a Json file using script task in SSIS在 SSIS 中使用脚本任务导入 Json 文件
【发布时间】：2021-01-07 21:06:24
【问题描述】：

我不熟悉在 SSIS 中使用脚本任务导入 API。我必须管理一个简单的 API JSON 文件的导入，但是第二个 API JSON 文件有点棘手。我一直在看代码，只是不知道我做错了什么。

我的 JSON 文件有一个标头，我需要确定我需要循环通过多少页 API 来获取数据，但我迷路的是，如何导入下一位数据。我想我已经很接近了，但由于我是新手，所以我可以在一些指导下做到这一点。

我在下面的脚本任务中的代码，已经看到流阅读器导入数据，但我不使用什么（例如列表、类或字典）来导入 countIn、countOut 等列。

    try
        {
            //Call getWebServiceResult to return our WorkGroupMetric array
            WorkGroupMetric[] outPutMetrics = GetWebServiceResult(wUrl);

            //For each group of metrics output records
            foreach (var metric in outPutMetrics)
            {
                APIBuffer.AddRow();
                APIBuffer.count = metric.count;
                APIBuffer.currentpage =  metric.currentpage;
                APIBuffer.totalpages = metric.totalpages;
                APIBuffer.countIn = metric.result.countIn;
                APIBuffer.countOut = metric.result.countOut;
                APIBuffer.type = metric.result.type;
                APIBuffer.countLine = metric.result.countLine;
                APIBuffer.from = metric.result.from;
                APIBuffer.to = metric.result.to;
            }

        }
        catch (Exception e)
        {
            FailComponent(e.ToString());
        }
    }

    private WorkGroupMetric[] GetWebServiceResult(string wUrl)
    {
        HttpWebRequest httpWReq = (HttpWebRequest)WebRequest.Create(wUrl);
        HttpWebResponse httpWResp = (HttpWebResponse)httpWReq.GetResponse();
        WorkGroupMetric[] jsonResponse = null;
        try
        {
            //Test the connection
            if (httpWResp.StatusCode == HttpStatusCode.OK)
            {

                Stream responseStream = httpWResp.GetResponseStream();
                string jsonString = null;

                //Set jsonString using a stream reader
                using (StreamReader reader = new StreamReader(responseStream))
                {
                    jsonString = reader.ReadToEnd().Replace("\\", "");
                    reader.Close();
                }

                //Deserialize our JSON
                JavaScriptSerializer sr = new JavaScriptSerializer();
                //JSON string comes in with a leading and trailing " that need to be removed for parsing to work correctly
                //The JSON here is serialized weird, normally you would not need this trim
                jsonResponse = sr.Deserialize<WorkGroupMetric[]>(jsonString.Trim('"'));

            }
            //Output connection error message
            else
            {
                FailComponent(httpWResp.StatusCode.ToString());

            }
        }

        //Output JSON parsing error
        catch (Exception e)
        {
            FailComponent(e.ToString());
        }
        return jsonResponse;

        throw new NotImplementedException();
    }

    private void FailComponent(string errorMsg)
    {
        bool fail = false;
        IDTSComponentMetaData100 compMetadata = this.ComponentMetaData;
        compMetadata.FireError(1, "Error Getting Data From Webservice!", errorMsg, "", 0, out fail);

    }
}
//Class to hold our workgroup metrics
class WorkGroupMetric
{
    public string count { get; set; }
    public string currentpage { get; set; }
    public string totalpages  { get; set; }
    public List<Result> result { get; set; }   

}

class Result
{
    public string countIn { get; set; }
    public string countOut { get; set; }
    public string type { get; set; }
    public string countLine { get; set; }
    public string from { get; set; }
    public string to { get; set; }

}

【问题讨论】：

如果您的 SSIS 包是 80% 的 C# 代码，那么 SSIS 不是正确的工具。无论如何，如何导入下一位数据是指如何调用下一页，或者如何访问results 数组中的数据？
您的代码是否按原样工作？你试过循环metric.results吗？它是有效的还是你得到一个错误？
您好，尼克，感谢您的回复。这个项目是为了我的工作，他们正在使用 SSIS atm，出于好奇，什么是正确的微软工具，因为他们目前微软很重。 - 基兰
所以我得到的错误是public List<Result> result { get; set; } 行。错误是List<Result>' does not contain a definition for 'countIn' and no accessible extension method 'countIn' accepting a first argument of type 'List<Result>' could be found (are you missing a using directive or an assembly reference?
然后我将public List<Result> result { get; set; } 更改为public Result result { get; set; } 代码更快乐，但随后我在foreach (var metric in outPutMetrics) 中收到一条错误消息，说System.NullReferenceException HResult=0x80004003 Message=Object reference not set to an instance of an object. 我一无所知，我知道它与json 文件有关数据层次结构只是不知道如何解决它。

标签： c# arrays json ssis script-task

【解决方案1】：

所以我已经让代码工作了，发布它以便将来可以帮助某人。

   try
        {
            //Call getWebServiceResult to return our WorkGroupMetric array
            WorkGroupMetric outPutMetrics = GetWebServiceResult(wUrl);

            //For each group of metrics output records
            //foreach (var metric in outPutMetrics)
            //{


            var ts = outPutMetrics.results;

            totalcount = Int32.Parse(outPutMetrics.count);

            foreach (var a in ts)
            {
                Output0Buffer.AddRow();
                //Output0Buffer.count = outPutMetrics.count;
                
                Output0Buffer.countIn = a.countIn;
                Output0Buffer.countOut = a.countOut;
                Output0Buffer.type = a.type;
                Output0Buffer.countLine = a.countLine;
                Output0Buffer.from = a.from;
                Output0Buffer.to = a.to;

                i = i + 1;
            }

            Console.Write(i);
        }
        catch (Exception e)
        {
            FailComponent(e.ToString());
        }
    }

    private WorkGroupMetric GetWebServiceResult(string wUrl)
    {
        HttpWebRequest httpWReq = (HttpWebRequest)WebRequest.Create(wUrl);
        HttpWebResponse httpWResp = (HttpWebResponse)httpWReq.GetResponse();
        WorkGroupMetric jsonResponse = null;
        try
        {
            //Test the connection
            if (httpWResp.StatusCode == HttpStatusCode.OK)
            {

                Stream responseStream = httpWResp.GetResponseStream();
                string jsonString = null;

                //Set jsonString using a stream reader
                using (StreamReader reader = new StreamReader(responseStream))
                {
                    jsonString = reader.ReadToEnd().Replace("\\", "");
                    reader.Close();
                }

                //Deserialize our JSON
                JavaScriptSerializer sr = new JavaScriptSerializer();
                //JSON string comes in with a leading and trailing " that need to be removed for parsing to work correctly
                //The JSON here is serialized weird, normally you would not need this trim
                jsonResponse = sr.Deserialize<WorkGroupMetric>(jsonString.Trim('"'));

            }
            //Output connection error message
            else
            {
                FailComponent(httpWResp.StatusCode.ToString());

            }
        }

        //Output JSON parsing error
        catch (Exception e)
        {
            FailComponent(e.ToString());
        }
        return jsonResponse;

        throw new NotImplementedException();
    }

    private void FailComponent(string errorMsg)
    {
        bool fail = false;
        IDTSComponentMetaData100 compMetadata = this.ComponentMetaData;
        compMetadata.FireError(1, "Error Getting Data From Webservice!", errorMsg, "", 0, out fail);

    }
}
//Class to hold our work group metrics
class WorkGroupMetric
{
    public string count { get; set; }
    public string currentpage { get; set; }
    public string totalpages { get; set; }
    public Result[] results { get; set; }

}

class Result
{
    public string countIn { get; set; }
    public string countOut { get; set; }
    public string type { get; set; }
    public string countLine `enter code here`{ get; set; }
    public string from { get; set; }
    public string to { get; set; }

}

【讨论】：

感谢您发布完整的代码。我看到我的所有建议都被采纳了。

【解决方案2】：

通过这个例子进行

https://www.tomasvera.com/programming/using-javascriptserializer-to-parse-json-objects/

这可能应该是您的 C# 类定义，使用数组而不是 List

class WorkGroupMetric
{
    public string count { get; set; }
    public string currentpage { get; set; }
    public string totalpages  { get; set; }
    public Result[] results { get; set; }   

}

class Result
{
    public string countIn { get; set; }
    public string countOut { get; set; }
    public string type { get; set; }
    public string countLine { get; set; }
    public string from { get; set; }
    public string to { get; set; }

}

此外，您每次调用只会获得一个 WorkGroupMetric，而不是它们的数组，因此您可能应该使用这个：

WorkGroupMetric

不是这个

WorkGroupMetric[]

整个

【讨论】：

嗨尼克，非常感谢你的帮助，它并没有解决我的问题，但它肯定指出了寻找答案的正确方向。
好东西。如果您想出答案，很高兴在这里发布以帮助其他人（包括我！）