日志文件两行之间的时间差答案

【问题标题】：Difference of Time between two lines of a log file日志文件两行之间的时间差
【发布时间】：2011-12-23 18:03:05
【问题描述】：

如果这太基本了，我很抱歉。我真的不是在找人来做这项工作，而是为我指明正确的方向。我有一个可以追溯到几年前的日志文件，我想从中提取信息以确定在性能缓慢时查找模式需要多长时间。我能够阅读每一行，但无法阅读上一行来获取时间。

日志文件如下：

~
Other Stuff
~
12/21/11 18:58:15 Inserting data into ST_ITEMS
ST_ITEMS Row: 10000 inserted at 12/21/11 19:40:06
ST_ITEMS Row: 20000 inserted at 12/21/11 20:05:58
ST_ITEMS Row: 30000 inserted at 12/21/11 20:37:03
ST_ITEMS Row: 40000 inserted at 12/21/11 20:59:25
ST_ITEMS Row: 50000 inserted at 12/21/11 21:17:43
ST_ITEMS Row: 60000 inserted at 12/21/11 21:54:47
12/21/11 21:59:24 Finished inserting data into  Staging Tables

~
Other Stuff
~

12/21/11 22:04:43 Inserting data into ST_ITEMS
ST_ITEMS Row: 10000 inserted at 12/21/11 22:38:53
ST_ITEMS Row: 20000 inserted at 12/21/11 23:06:33
ST_ITEMS Row: 30000 inserted at 12/21/11 23:33:03
ST_ITEMS Row: 40000 inserted at 12/22/11 00:05:38
ST_ITEMS Row: 50000 inserted at 12/22/11 00:45:59
ST_ITEMS Row: 60000 inserted at 12/22/11 01:12:42
ST_ITEMS Row: 70000 inserted at 12/22/11 01:40:02
ST_ITEMS Row: 80000 inserted at 12/22/11 02:14:23
ST_ITEMS Row: 90000 inserted at 12/22/11 03:04:15
ST_ITEMS Row: 100000 inserted at 12/22/11 03:47:13
ST_ITEMS Row: 110000 inserted at 12/22/11 04:36:21
ST_ITEMS Row: 120000 inserted at 12/22/11 05:44:47
ST_ITEMS Row: 130000 inserted at 12/22/11 06:28:24
ST_ITEMS Row: 140000 inserted at 12/22/11 07:10:55
ST_ITEMS Row: 150000 inserted at 12/22/11 07:35:16
12/22/11 07:40:28 Finished inserting data into  Staging Tables

~
Other Stuff
~

基本上，我想通过从它上面的行中减去一行的日期/时间来计算每 10000 行需要多长时间。我将 Perl 和 Bash 视为选项，但似乎 Perl 提供了更多可能性。

PERL #!/usr/bin/perl

use strict;
use warnings;

use Date::Parse;
use Date::Format;

my $start = "2007-11-17 12:50:22";
my $stop  = "2007-11-17 12:53:22";
my $diff  = str2time($stop) - str2time($start);

#printf "diff between %s and %s is %d seconds\n", $start, $stop, $diff;

open(LOG,"info_refresh_tvl.log.122111_185800") or die "Unable to open logfile:$!\n";
while(my $line = <LOG>){


        if ($line=~/inserted\b/)

        {
        #Pseudocode  
            #Parse time from Pervious Line
            #Parse time from Current Line
            #Calculate Difference of Time
                    #my $diff  = str2time($stop) - str2time($start);
            #printf "diff between %s and %s is %d seconds\n", $start, $stop,     $diff; ')

            printf $line ;


        }

}
close(LOG);

重击

grep 'ST_ITEMS Row:' logfile122111.log | while read line
   do
        event=$(echo "$line" | awk '{print $6 " " $7}')

       case $event in
          "10000")
                ;;
          *)
                past=$(echo "$line" | awk '{print $6 " " $7}')
                current=$(echo "$line" | awk '{print $6 " " $7}'
                echo $past
                echo $current)
                ;;
       esac



echo $event


   done

【问题讨论】：

我正在考虑将信息放入时间数组中，然后从数组中解析时间并减去。 while(my $line = ){ if ($line=~/inserted\b/) { push(@times, $line) ; } 打印“@times”； } 关闭（日志）；

标签： perl shell sh

【解决方案1】：

比较后继续时只需保存每一行。完成后用当前行覆盖它。

在伪代码中：

$CurrentLine = $line;
#Parse time from $CurrentLine
#Parse time from $LastLine
#Calculate difference of time
$LastLine = $line;

【讨论】：

【解决方案2】：

正如其他人已经提到的，只是保留以前的时间以供参考。这是使用 Time::Piece 的快速示例，它是 perl 5.10 以来的核心模块：

use Time::Piece;

my $lasttime;
while(<DATA>) {
    chomp;

    my $diff;
    if(m{(\d+/\d+/\d+ \d+:\d+:\d+)}) {
        my $t = Time::Piece->strptime($1, "%D %H:%M:%S");
        if(defined $lasttime) {
            $diff = $t - $lasttime;
        }
        $lasttime = $t;
    }
    undef $lasttime if m{Finished inserting data};

    print "$_\t", ($diff && $diff->pretty) , "\n";
}

__DATA__
~
Other Stuff
~
12/21/11 18:58:15 Inserting data into ST_ITEMS
ST_ITEMS Row: 10000 inserted at 12/21/11 19:40:06
ST_ITEMS Row: 20000 inserted at 12/21/11 20:05:58
ST_ITEMS Row: 30000 inserted at 12/21/11 20:37:03
ST_ITEMS Row: 40000 inserted at 12/21/11 20:59:25
...

打印

~   
Other Stuff 
~   
12/21/11 18:58:15 Inserting data into ST_ITEMS  
ST_ITEMS Row: 10000 inserted at 12/21/11 19:40:06   41 minutes, 51 seconds
ST_ITEMS Row: 20000 inserted at 12/21/11 20:05:58   25 minutes, 52 seconds
ST_ITEMS Row: 30000 inserted at 12/21/11 20:37:03   31 minutes, 5 seconds
ST_ITEMS Row: 40000 inserted at 12/21/11 20:59:25   22 minutes, 22 seconds

【讨论】：

【解决方案3】：

grep -B1 gets the previous line before the line that is currently matched

【讨论】：

【解决方案4】：

您可以定义两个变量来保持时间。在伪代码中会给出：

my $old = undef;
my $current;

while (my $line = <LOG>) {
    $line =~ /inserted at (.*)/ or next;
    $current = parse_time($1);
    if (defined $old) {
        printf("Time to insert 10k rows: %d\n", datediff($current, $old));
    }
    $old = $current;
}

（填补parse_time()和datediff()的空白，你应该被设置）

【讨论】：