带有 .NET 的 AWS - 从 CloudWatch 读取日志 - 没有返回日志数据答案

【问题标题】：AWS with .NET - reading logs from CloudWatch - no log data returned带有 .NET 的 AWS - 从 CloudWatch 读取日志 - 没有返回日志数据
【发布时间】：2020-03-02 06:27:36
【问题描述】：

我正在尝试使用 .NET 从 CloudWatch 读取使用 SNS 发送的消息的日志数据。

从 CloudWatch 控制台（CloudWatch \ CloudWatch Logs \ Logs Insights）我输入：

Date range: custom (2w)
LogGroup: sns/ap...../8...../LogName 
Query: fields @timestamp, @message | sort @timestamp desc | limit 20

它返回大量日志记录（@timestamp | @message）

我正在尝试使用 .net AWS SDK 从 c# 做同样的事情：

public async Task GetLogs()
{
    string logGroupName = "sns/ap...../8...../LogName";

    AWSOptions options = configuration.GetAWSOptions();
    IAmazonCloudWatchLogs logs = options.CreateServiceClient<IAmazonCloudWatchLogs>();

    StartQueryRequest startQueryRequest = new StartQueryRequest();
    startQueryRequest.LogGroupName = logGroupName;
    startQueryRequest.StartTime = 1577850562; //1 Jan 2020
    TimeSpan t = DateTime.UtcNow - new DateTime(1970, 1, 1); //Epoch time starts on 1/1/1970
    int secondsSinceEpoch = (int)t.TotalSeconds;
    startQueryRequest.EndTime = secondsSinceEpoch;
    startQueryRequest.QueryString = "fields @timestamp, @message | sort @timestamp desc";
    startQueryRequest.Limit = 1000;
    StartQueryResponse response2 = await logs.StartQueryAsync(startQueryRequest);

    Console.WriteLine();
}

不知道为什么它不返回任何记录。

Response2 = 
   ContentLength: 50
   QueryId: "guid..."
   ResponseMetadata:
      Metadata Count = 0
      RequestId = "guid..."

知道我做错了什么吗？谢谢！

【问题讨论】：

标签： amazon-web-services .net-core amazon-cloudwatchlogs aws-cloudwatch-log-insights

【解决方案1】：

如果它对其他人有帮助 - 这是我使用 C# 从 AWS 获取日志的方法

public async Task GetLogs()
{
    string logGroupName = "log group name from CloudWatch Log";

    AWSOptions options = configuration.GetAWSOptions();
    IAmazonCloudWatchLogs logs = options.CreateServiceClient<IAmazonCloudWatchLogs>();

    StartQueryRequest startQueryRequest = new StartQueryRequest();
    startQueryRequest.LogGroupName = logGroupName;
    startQueryRequest.StartTime = 1577850562; //1 Jan 2020
    TimeSpan t = DateTime.UtcNow - new DateTime(1970, 1, 1); //Epoch time starts on 1/1/1970
    int secondsSinceEpoch = (int)t.TotalSeconds;
    startQueryRequest.EndTime = secondsSinceEpoch;
    startQueryRequest.QueryString = "fields @timestamp, @message | sort @timestamp desc";
    startQueryRequest.Limit = 1000;
    StartQueryResponse startQueryResponse = await logs.StartQueryAsync(startQueryRequest);

    GetQueryResultsRequest getQueryRequest = new GetQueryResultsRequest();
    getQueryRequest.QueryId = startQueryResponse.QueryId;

    GetQueryResultsResponse getQueryResults = await logs.GetQueryResultsAsync(getQueryRequest);
    for (var i=0; i<getQueryResults.Results.Count; i++)
    {
        ResultField timestampResult = getQueryResults.Results[i][0];
        ResultField messageResult = getQueryResults.Results[i][1];
        var message = JsonConvert.DeserializeObject(messageResult.Value);
        Console.WriteLine(message);
    }           
    Console.WriteLine();
}

【讨论】：

您还应该检查结果的状态：“如果输出中 Status 字段的值是 Running，则此操作仅返回部分结果。如果您看到状态值为 Scheduled 或 Running，您可以稍后重试操作以查看最终结果。"
我已经掌握了中等水平的 AWS C# 技能，虽然这看起来像花哨的代码，但我什至无法真正开始使用它。我可以在控制台 Cloudwatch 中运行过滤器。我想用 C# 来做，但我什至不知道命名空间。我添加了 AWS.Core 和 CloudWatch NuGet 包，但没有任何进展。任何建议将不胜感激。

【解决方案2】：

您只是开始查询，需要调用GetQueryResults获取数据：https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_GetQueryResults.html

【讨论】：

【解决方案3】：

我有一个没有拼写错误的工作示例，如果查询需要一些时间才能完成，它将重试：

            const string LogGroupName = "<your log group name here>";
            const string QueryString = "<your query string here>";

            var epoch = new DateTime( 1970, 1, 1 );
            var s = DateTime.UtcNow.AddMinutes( -15 ) - epoch;
            var e = DateTime.UtcNow - epoch;

            using( var client = new AmazonCloudWatchLogsClient() )
            {
                var startQueryRequest = new StartQueryRequest
                {
                    LogGroupName = LogGroupName,
                    StartTime = ( long ) s.TotalSeconds,
                    EndTime = ( long ) e.TotalSeconds,
                    QueryString = QueryString,
                    Limit = 1000
                };

                var startQueryResponse = await client.StartQueryAsync( startQueryRequest );
                var resultRequest = new GetQueryResultsRequest { QueryId = startQueryResponse.QueryId };
                var result = await client.GetQueryResultsAsync( resultRequest );

                // Retry loop with a timeout (AWS states that queries can run up to 15 minutes)
                for( var i = 0 ; i < 60 && ( result.Status == QueryStatus.Running || result.Status == QueryStatus.Scheduled ) ; i++ )
                {
                    System.Threading.Thread.Sleep( 250 );
                    result = await client.GetQueryResultsAsync( resultRequest );
                }

                // Query failed or timed out
                if( result.Status != QueryStatus.Complete )
                {
                    throw new Exception( "Could not complete query." );
                }

                foreach( var item in result.Results )
                {
                    // Use the results here
                }
            }

【讨论】：