【问题标题】:how to fetch logs content between two dates in windows powershell?如何在 Windows Powershell 中的两个日期之间获取日志内容?
【发布时间】:2019-12-10 18:20:26
【问题描述】:

在我们的项目中,我们正在获取 AppDynamics 日志(应用程序日志)和机器日志,有时日志的大小会增加,这会耗尽磁盘大小。我要做的是获取两个日期之间的内容,例如 11 月 10 日和 11 月 13 日,然后删除其余日期。由于我们在windows环境下工作,所以需要在powershell中完成。在 linux 中处理这些事情更容易,但我不擅长 powershell 脚本。下面是代码sn-p。

[AD Thread-Metric Reporter1] 10 Nov 2019 14:47:32,899 ERROR ManagedMonitorDelegate - Error sending metrics - will requeue for later transmission
com.singularity.ee.agent.commonservices.metricgeneration.metrics.MetricSendException: Connection back off limitation in effect: /controller/instance/702/metrics
    at com.singularity.ee.agent.commonservices.metricgeneration.AMetricSubscriber.publish(AMetricSubscriber.java:350)
    at com.singularity.ee.agent.commonservices.metricgeneration.MetricReporter.run(MetricReporter.java:113)
    at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run` ]
[AD Thread-Metric Reporter1] 11 Nov 2019 14:46:32,899 ERROR ManagedMonitorDelegate - Error sending metrics - will requeue for later transmission
com.singularity.ee.agent.commonservices.metricgeneration.metrics.MetricSendException: Connection back off limitation in effect: /controller/instance/702/metrics
    at com.singularity.ee.agent.commonservices.metricgeneration.AMetricSubscriber.publish(AMetricSubscriber.java:350)
    at com.singularity.ee.agent.commonservices.metricgeneration.MetricReporter.run(MetricReporter.java:113)
    at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run` ]
[extension-scheduler-pool-5] 13 Nov 2019 18:45:40,634  INFO ReportMetricsConfigSupplier - Basic metrics will be collected and reported through the SIM extension because SIM is enabled.
[extension-scheduler-pool-8] 14 Nov 2019 18:47:18,650  INFO ReportMetricsConfigSupplier - Basic metrics will be collected and reported through the SIM extension because SIM is enabled.` ]

带有文件路径的代码片段

# Get Start Time
$startDTM = (Get-Date)
$zstart = Read-Host -prompt '
Enter your start date in "10 Nov" format. Start date must be earlier than stop date.'
$zstop = Read-Host -prompt 'Enter your stop date in "13 Nov" format.  Stop date must be later than start date.'
# $zstart = '10 Nov'
# $zstop = '13 Nov'
$zstart= Select-String $zstart "$env:userprofile\Desktop\machine-log.txt" | Select-Object -ExpandProperty LineNumber
$zstop= Select-String $zstop "$env:userprofile\Desktop\machine-logtxt" | Select-Object -ExpandProperty LineNumber
$AppLog = gc $env:userprofile\Desktop\machine-log.txt 
$i = 0
$array = @()
foreach ($line in $AppLog){
foreach-object { $i++ }
if (($i -ge $zstart) -and ($i -le $zstop))
{$array += $line}}
$array | Out-File -encoding ascii -filepath $env:userprofile\Desktop\logfile-output.txt

执行脚本时遇到的错误。

   8923 8924 8925 8926 8927 8928 8929 8930 8931 8932 8933 8934 8935 8936 8937 8938 8939 8940 8941 8942 8943 
8944 8945 8946 8947 8948". Error: "Cannot convert the "System.Object[]" value of type "System.Object[]" to 
type "System.Int32"."
At C:\Users\xa_abbasmn\Documents\Logs\test.ps1:13 char:5
+ if (($i -ge $zstart) -and ($i -le $zstop))
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : ComparisonFailure

Could not compare "18" to "1 15 16 17 31 45 46 47 48 62 76 77 91 105 106 107 121 135 136 137 151 152 196 210 
211 225 239 240 241 255 269 270 271 285 299 300 314 315 329 330 331 332 346 390 404 447 448 449 463 477 478 
492 506 507 508 522 536 537 538 552 566 567 581 582 583 627 641 642 643 657 671 685 686 687 

powershell和windows版本: 名称:Windows PowerShell ISE 主机 - 版本:5.1.14409.1018 名称:Microsoft Windows Server 2012 R2 Standard 64bit

我们将非常感激您的帮助。 最好的问候,

【问题讨论】:

  • 在您的问题中添加以下内容后向我发表评论:什么版本的 windows? 32 位还是 64 位?什么版本的powershell?哪两个日期?
  • 重要的是要识别,该行下面的日志条目属于同一个条目,然后用新的日期识别。
  • 在您的问题中添加以下内容后向我发表评论:日志文件的完整路径/名称是什么?
  • @somebadhat 文件的完整路径是 C:\logs\machine-logs.log
  • 在对问题进行以下更改后给我评论:将日志文件的路径添加到问题中。从您在尝试使用我的答案失败时使用的日志文件中获取代码 sn-p(11 月 9 日至 11 月 14 日)并替换您问题中的代码 sn-p。

标签: windows powershell logging powershell-5.0


【解决方案1】:

Windows 10 64 位。 Powershell 5. 不需要管理员权限。

如何使用powershell 5快速有效地从输入1到输入2的大型日志文件中提取文本?

下面的示例日志文件。

出于测试目的,将您的日志文件复制到桌面并将其命名为 logfile.txt

Windows Server 2012 R2 Standard 64 位的默认文本编辑器是什么?见第 42 行。 你有什么程序与 .txt 文件相关联?见第 42 行。

# Get Start Time
$startDTM = (Get-Date)
$zstart = Read-Host -prompt '
Enter your start date in "10 Nov" format (w/o quotes). Start date must be earlier than stop date.'
$zstop = Read-Host -prompt 'Enter your stop date in "13 Nov" format (w/o quotes).  Stop date must be later than start date.'
# $zstart = '10 Nov'
# $zstop = '13 Nov'
$zstart= Select-String $zstart "$env:userprofile\Desktop\logfile.txt" | Select-Object -ExpandProperty LineNumber
$zstop= Select-String $zstop "$env:userprofile\Desktop\logfile.txt" | Select-Object -ExpandProperty LineNumber
$AppLog = gc $env:userprofile\Desktop\logfile.txt 
$i = 0
$array = @()
foreach ($line in $AppLog){
foreach-object { $i++ }
if (($i -ge $zstart) -and ($i -le $zstop))
{$array += $line}}
$array | Out-File -encoding ascii -filepath $env:userprofile\Desktop\logfile-edited.txt 
# begin get file size 
$y = (Get-ChildItem "$env:userprofile\Desktop\logfile.txt" | Measure-Object -property length -sum)
$y = [System.Math]::Round(($y.Sum /1KB),2) 
$z = (Get-ChildItem "$env:userprofile\Desktop\logfile-edited.txt" | Measure-Object -property length -sum)
$z = [System.Math]::Round(($z.Sum /1KB),2) 
#   end get file size 
# get Stop Time
$endDTM = (Get-Date) 
Write-Host "
Extracted $z KB from $y KB. The extraction took $(($endDTM-$startDTM).totalseconds) seconds"
start-process -wait notepad $env:userprofile\Desktop\logfile-edited.txt
remove-item $env:userprofile\Desktop\logfile-edited.txt 
exit 

日志文件.txt:

[AD Thread-Metric Reporter1] 09 Nov 2019 14:48:32,899 ERROR ManagedMonitorDelegate - Error sending metrics
com.singularity.ee.agent.commonservices.metricgeneration.metrics.MetricSendException: Connection back off limitation
at com.singularity.ee.agent.commonservices.metricgeneration.AMetricSubscriber.publish(AMetricSubscriber.java:350)
at com.singularity.ee.agent.commonservices.metricgeneration.MetricReporter.run(MetricReporter.java:113)
at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run ]

[AD Thread-Metric Reporter1] 10 Nov 2019 14:47:32,899 ERROR ManagedMonitorDelegate - Error sending metrics
com.singularity.ee.agent.commonservices.metricgeneration.metrics.MetricSendException: Connection back off limitation
at com.singularity.ee.agent.commonservices.metricgeneration.AMetricSubscriber.publish(AMetricSubscriber.java:350)
at com.singularity.ee.agent.commonservices.metricgeneration.MetricReporter.run(MetricReporter.java:113)
at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run ]

[AD Thread-Metric Reporter1] 11 Nov 2019 14:46:32,899 ERROR ManagedMonitorDelegate - Error sending metrics
com.singularity.ee.agent.commonservices.metricgeneration.metrics.MetricSendException: Connection back off limitation
at com.singularity.ee.agent.commonservices.metricgeneration.AMetricSubscriber.publish(AMetricSubscriber.java:350)
at com.singularity.ee.agent.commonservices.metricgeneration.MetricReporter.run(MetricReporter.java:113)
at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run ]

[extension-scheduler-pool-5] 13 Nov 2019 18:45:40,634  INFO ReportMetricsConfigSupplier - Basic metrics will be collected 
[extension-scheduler-pool-8] 14 Nov 2019 18:47:18,650  INFO ReportMetricsConfigSupplier - Basic metrics will be collected

在 logfile.txt 上运行脚本的结果:

[AD Thread-Metric Reporter1] 10 Nov 2019 14:47:32,899 ERROR ManagedMonitorDelegate - Error sending metrics
com.singularity.ee.agent.commonservices.metricgeneration.metrics.MetricSendException: Connection back off limitation
at com.singularity.ee.agent.commonservices.metricgeneration.AMetricSubscriber.publish(AMetricSubscriber.java:350)
at com.singularity.ee.agent.commonservices.metricgeneration.MetricReporter.run(MetricReporter.java:113)
at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run ]

[AD Thread-Metric Reporter1] 11 Nov 2019 14:46:32,899 ERROR ManagedMonitorDelegate - Error sending metrics
com.singularity.ee.agent.commonservices.metricgeneration.metrics.MetricSendException: Connection back off limitation
at com.singularity.ee.agent.commonservices.metricgeneration.AMetricSubscriber.publish(AMetricSubscriber.java:350)
at com.singularity.ee.agent.commonservices.metricgeneration.MetricReporter.run(MetricReporter.java:113)
at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run ]

[extension-scheduler-pool-5] 13 Nov 2019 18:45:40,634  INFO ReportMetricsConfigSupplier - Basic metrics will be collected 

Powershell in four hours at Youtube

Parse and extract text with powershell at Bing

如何使用 powershell 5 从大型日志文件中从第 x 行到第 y 行快速有效地提取文本?

如何使用 powershell 5 从 date1 到 date2 的大型日志文件中快速有效地提取文本?

【讨论】:

  • 非常感谢@somebadhat 的简短解释。当我按照您的指示设置代码时,我收到以下问题“以“11 月 10 日”格式输入您的开始日期。开始日期必须早于停止日期。:“11 月 10 日”以“11 月 13 日”格式输入您的停止日期. 停止日期必须晚于开始日期。: "13 Nov" -> 从 2608.54 KB 中提取了 0 KB。提取时间为 16.8283796 秒"
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2018-01-11
  • 1970-01-01
  • 2017-04-02
  • 2020-03-02
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多