awk sed perl 根据关键字匹配合并行

Question

由于我有限的 awk/sed 魔法，我一直在这个问题上碰壁。我很乐意使用 awk、sed、bash、perl 或任何其他工具来完成此文本操作。

我有以下输出，并希望根据某种键匹配合并行：

 Node: server1
 Active Server: SECONDARY
 Standby Server: PRIMARY
 Primary 192.168.1.1
 Secondary 192.168.1.2

 Node: server2
 Active Server: PRIMARY
 Standby Server: SECONDARY
 Primary 10.1.1.1
 Secondary 10.1.1.2

期望的输出：

 Node: server1
 Active Server: Secondary 192.168.1.2
 Standby Server: Primary 192.168.1.1

 Node: server2
 Active Server: Primary 10.1.1.1
 Standby Server: Secondary 10.1.1.2

所以我需要根据“主要”和“次要”这两个词来合并这些行。我的第一个想法是将“Primary”更改为“PRIMARY”，这样更容易匹配。

我的最终目标是：

 server1,Active,192.168.1.2,Standby,192.168.1.1
 server2,Active,10.1.1.1,Standy,10.1.1.2

（但我可以在帮助合并行后弄清楚这部分）

感谢您的帮助！

Answer 1

你可以用这个awk

awk -v RS="" '{$5=tolower($5);sub(".",substr(toupper($5),1,1),$5);$8=tolower($8);sub(".",substr(toupper($8),1,1),$8);print $1,$2"\n"$3,$4,$5,$10"\n",$6,$7,$8,$12}' file
Node: server1
Active Server: Secondary 192.168.1.1
 Standby Server: Primary 192.168.1.2
Node: server2
Active Server: Primary 10.1.1.1
 Standby Server: Secondary 10.1.1.2

通过将RS 设置为空，awk 与行组一起使用。

Answer 2

它是密集且非常丑陋的多线，

perl -00 -nE'
  s/ ^(\w+)\s+([\d.]+)\s* / $s{$1}=$2; ""/xmge;
  ($l=$_) =~ s! \s*\w+:\s*|\n !,!xg;
  $l =~ s|\U$_|$s{$_}| for keys %s;
  ($_=$l) =~ s/^,|,$//g;
  say
' file

输出

server1,Active,192.168.1.2,Standby,192.168.1.1
server2,Active,10.1.1.1,Standby,10.1.1.2

说明

# -00 => instead of single line read lines into $_ until \n\n+
perl -00 -nE'
  # read and remove 'Primary|Secondary IP' into $s{Primary} = IP
  s/ ^(\w+)\s+([\d.]+)\s* / $s{$1}=$2; ""/xmge;

  # replace 'something:' or new line by ','
  ($l=$_) =~ s! \s*\w+:\s*|\n !,!xg;

  # replace SECONDARY|PRIMARY with actual IP address
  $l =~ s|\U$_|$s{$_}| for keys %s;

  # remove ',' at beginning and end of the string
  ($_=$l) =~ s/^,|,$//g;

  # print result
  say
' file

Answer 3

或者使用单线作为中间所需的解决方案（最终解决方案遵循）：

perl -00 -lpe '
     s/ Server: \K(\w+)(?=.*^(\1[^\n]*))/$2/ismg;
     s/\n[^:]+$//;
   ' file.txt

输出：

Node: server1
Active Server: Secondary 192.168.1.2
Standby Server: Primary 192.168.1.1

Node: server2
Active Server: Primary 10.1.1.1
Standby Server: Secondary 10.1.1.2

解释：

• 开关：
  • -00: 以段落模式处理输入（由双回车分隔）
  • -l: 开启行尾处理
  • -p：假设"while (<>) { ...; print; }" 循环程序
  • -e: 评估 perl 代码
• 代码：
  • 将所有服务器值替换为以相同键开头的匹配行
  • 删除底部的服务器列表。

为了获得您想要的最终解决方案，以下一个班轮将实现该目标。

与第一个解决方案相比有一些细微的变化，例如使用-n 而不是-p，因为我们希望从记录之间的两个换行符移动到一个新行。但是，正则表达式工具是相同的：

perl -00 -ne'
    s/ Server: (\w+)(?=.*^\1\s+(\S+))/:$2/ismg;
    s/\n[^:]+$//;
    s/^Node: //;
    s/[\n:]/,/g;
    print "$_\n";
  ' file.txt

输出：

server1,Active,192.168.1.2,Standby,192.168.1.1
server2,Active,10.1.1.1,Standby,10.1.1.2

Answer 4

这个 Perl 解决方案似乎可以满足您的要求。它只是将值逐行提取到哈希中，并在所有需要的值都存在时转储哈希内容。

更新我使用any 来自List::Util 代替grep 以使代码更清晰。

use strict;
use warnings;
use autodie;

use List::Util 'any';

my @names = qw/ node active standby primary secondary /;

open my $fh, '<', 'myfile.txt';

my %server;

while (my $line = <$fh>) {
  next unless my ($key, $val) = lc($line) =~ /(\w+).*\s+(\S+)/;

  %server = () if $key eq 'server';
  $server{$key} = $val;

  unless ( any { not exists $server{$_} } @names ) {
    printf "%s,Active,%s,Standby,%s\n", @server{'node', $server{active}, $server{standby}};
    %server = ();
  }
}

输出

server1,Active,192.168.1.2,Standby,192.168.1.1
server2,Active,10.1.1.1,Standby,10.1.1.2

Answer 5

再详细一点：

use strict;
use warnings;
use feature qw/say/;

my $struct;
local $/ = 'Node: ';


for my $record (<DATA>) {
    next if $record =~ /^Node:/; # skip first
    my ($node, @values) = split /\n\s*/, $record;
    for my $line (@values) { 
        my ($intent, $actual, $ip);
        if ( ($intent, $actual) = $line =~ /(Active|Standby) Server: (.*)$/ ) {
            $struct->{$node}{lc($intent)} = lc($actual);
        }
        elsif ( ($actual, $ip) = $line  =~ /(Primary|Secondary) (.*)$/ ) {
            $struct->{$node}{lc($actual)} = $ip;
        }
    }
}


for my $node (sort keys %$struct) {
    printf "Node: %s\n", $node;
    printf "Active server: %s %s\n", ucfirst $struct->{$node}{active}, $struct->{$node}{$struct->{$node}{active}};
    printf "Standby server: %s %s\n", ucfirst $struct->{$node}{standby}, $struct->{$node}{$struct->{$node}{standby}};
    print "\n";
}

## Desired final output is simpler:
for my $node (sort keys %$struct) {
    say join ',', $node, 'Active', $struct->{$node}{$struct->{$node}{active}}, 'Standby', $struct->{$node}{$struct->{$node}{standby}};
}


__DATA__
Node: server1
 Active Server: SECONDARY
 Standby Server: PRIMARY
 Primary 192.168.1.1
 Secondary 192.168.1.2

 Node: server2
 Active Server: PRIMARY
 Standby Server: SECONDARY
 Primary 10.1.1.1
 Secondary 10.1.1.2

Answer 6

这是 awk 中的一个选项。

#!/usr/bin/awk -f

# Output processing goes in a function, as it's called from different places
function spew() {
  split(servers[d["active"]], active);
  split(servers[d["standby"]], standby);
  printf("%s,%s,%s,%s,%s\n",
     d["name"], active[1], active[2], standby[1], standby[2]);
}

# trim unnecessary (leading) whitespace
1 { $1=$1; }

# Store our references
$1=="Active" {
  d["active"]=tolower($3);
}
#
$1=="Standby" {
  d["standby"]=tolower($3);
}

# And store our data
/^ *[A-za-z]+ [0-9.]+$/ {
  servers[tolower($1)]=tolower($0);
}

# Then, if we hit a new record, process the last one.
$1=="Node:" && length(d["name"]) {
  spew();
}

# And if we've just process a record, clear our workspace.
$1=="Node:" {
  delete d;
  delete s;
  d["name"]=$2;
}

# Finally, process the last record.
END {
  spew();
}

与其他一些解决方案相比，它的一个优势是它可以处理“主要”和“次要”以外的名称。这个想法是，如果你有这样的数据：

Node: serverN
Active Server: starfleet
Standby Server: babylon5
starfleet 172.16.0.1
babylon5 172.16.0.2

Active/Standby 行将通过索引引用记录，而不是假设“主要”或“次要”。

我已将所有内容标准化为小写以便于处理，但您当然可以调整 tolower() 以适应。

Answer 7

awk ' s==0{print;s=1;next;}
      s==1{i=$0;s=2;next;}
      s==2{j=$0;s=3;next;}
      s==3{r1=$0;s=4;next;}
      s==4{r2=$0;
           sub(/SECONDARY/,r2,i);sub(/PRIMARY/,r1,j);
           sub(/SECONDARY/,r2,j);sub(/PRIMARY/,r1,i);
           s=5; print i;print j;next}
      s==5{s=0;print}' input.txt

输出：

 Node: server1
 Active Server:  Secondary 192.168.1.2
 Standby Server:  Primary 192.168.1.1

 Node: server2
 Active Server:  Primary 10.1.1.1
 Standby Server:  Secondary 10.1.1.2

打印当前输入部分的第一行，将接下来的四行存储在变量中，然后进行替换，然后打印结果。然后读取并打印空白行并重新开始下一节。

Answer 8

awk '
    $1 == "Active"  {active = tolower($NF); next} 
    $1 == "Standby" {standby = tolower($NF); next} 
    $1 == "Primary" {ip["primary"] = $0; next} 
    $1 == "Secondary" {
        ip["secondary"] = $0
        print "Active Server:",ip[active]
        print "Standby Server:",ip[standby]
        next
    }
    1
'

这假定“次要”行位于“块”的末尾。

实现你的下一个输出：

awk -v OFS="," '
    $1 == "Node:"   {node = $NF}
    $1 == "Active"  {active = tolower($NF)} 
    $1 == "Standby" {standby = tolower($NF)} 
    $1 == "Primary" {ip["primary"] = $2} 
    $1 == "Secondary" {
        ip["secondary"] = $2; 
        print node, "Active",ip[active],"Standup",ip[standby]
    }
'

---

回应jhill的评论：

awk -v RS="" -v OFS=, '{
    node = active = standby = ""
    delete ip
    for (i=1; i<NF; i++) {
        if      ($i == "Node:")     {node=$(++i)}
        else if ($i == "Active")    {active = tolower( $(i+=2) )}
        else if ($i == "Standby")   {standby = tolower( $(i+=2) )}
        else if ($i == "Primary")   {ip["primary"] = $(++i)}
        else if ($i == "Secondary") {ip["secondary"] = $(++i)}
    }
    print node, "Active", ip[active], "Standup", ip[standby]
}'

Answer 9

您可以使用tr 消除空格，然后将sed 放回正确的位置并使用perl 获得您想要的输出：

输入文件：

tiago@dell:/tmp$ cat file
 Node: server1
 Active Server: SECONDARY
 Standby Server: PRIMARY
 Primary 192.168.1.1
 Secondary 192.168.1.2

 Node: server2
 Active Server: PRIMARY
 Standby Server: SECONDARY
 Primary 10.1.1.1
 Secondary 10.1.1.2

脚本：

tiago@dell:/tmp$ cat test.sh 
#! /bin/bash

tr -d '\n' < $1 | sed -r 's/(Node:)/\n\1/g' |\
     perl -lne '
        /^\s+$/ && next;
        /Node:\s+(\w+.*?)\s/ && {$server=$1};
        /Active Server:\s+(\w+.*?)\s/ && {$active=$1};
        /Standby Server:\s+(\w+.*?)\s/ && {$standby=$1};
        /Primary\s+(\w+.*?)\s/ && {$pri=$1};
        /Secondary\s+(\w+.*?)\s/ && {$sec=$1};

        if ( "$active" eq "PRIMARY" ){
            $out="$server,Active,$pri,Standby,$sec";
        }else{
            $out="$server,Active,$sec,Standby,$pri";          
        }
        print $out;
    '

执行：

tiago@dell:/tmp$ bash test.sh  file 
server1,Active,192.168.1.2,Standby,192.168.1.1
server2,Active,10.1.1.1,Standby,192.168.1.2