如何打印匹配的行、后面的 3 行和匹配的 URL

Question

我尝试从 SMTP 邮件中提取文本信息，即：

1. 日期（例如：2019 年 10 月 9 日星期三 01:55:58 -0700 (PDT)
2. 发件人（例如：来自 xxx.yyy.com (zzz:com. [111.222.333.444])
3. 邮件中的网址（例如：http://some.thing）

这是一个输入示例：

Delivered-To: SOME@ADDRESS.COM
Received: by X.X.X.X with SMTP id SOMEID;
        Wed, 9 Oct 2019 01:55:58 -0700 (PDT)
X-Received: by X.X.X.X with SMTP id SOMEID;
        Wed, 09 Oct 2019 01:55:58 -0700 (PDT)
Return-Path: <SOME@ADDRESS.COM>
Received: from SOME.URL.COM (SOME.OTHER.URL.COM. [X.X.X.X])
        by SOME.THIRD.URL.COM with ESMTP id SOMEID
        for <SOME@ADDRESS.COM>;
        Wed, 09 Oct 2019 01:55:58 -0700 (PDT)
SOME_HTML
SOME_HTML
href="http://URL1"><img
SOME_HTML
src="http://URL2"
SOME_HTML

示例被故意截断，因为标题较长，但这是为了示例

我已经尝试过 sed 和 awk，我设法做了一些事情，但不是我想要的。

SED：

sed -e 's/http/\nhttp/g' -n -e '/Received: from/{h;n;n;n;H;x;s/\n \+/;/;p}' a.txt

第一个是将 URL 放在一个留置权上，但之后我没有设法使用它。 无论如何，这不是按顺序排列的。

AWK：

BEGIN{
    RS = "\n";
    FS = "";
}
/Received: from/{
    from = $0;
    getline;
    getline;
    getline;
    date = $0
}
/"\"https?://[^\"]+"/
{
    FS="\"";
    print $0;
}
END{
    print date";"from;
};

除了 URL 之外，此方法有效。 rexgexp 在单行中不起作用是。 我还尝试通过使用 NR+3 的值来寻找更优雅的日期方式，但没有奏效。

并以 csv 格式显示：

日期;发件人;URL1;URL2;...

我更喜欢纯 sed 或纯 awk，因为我认为我可以使用 grep、tail、sed 和 awk 来完成，但我想学习，我更喜欢其中一个或两个 :)

Answer 1

嗯，下面这个长长的 sed 脚本，里面有 cmets：

sed -nE '
/Received: from /{
    # hold mu line!
    h

    # ach, spagetti code, here we go again
    : notdate
    ${
        s/.*/ERROR: INVALID INPUT: DATE NOT FOUND/
        p
        q1
    }
    # the next line after the line ending with ; should be the date
    /;$/!{
        # so search for a line ending with ;
        n
        b notdate
    }
    # the next line is the date
    n
    # remove leading spaces
    s/^[[:space:]]*//
    # grab the Received: from line
    G
    # and save it for later
    h
}

# headers end with an empty line
/^$/{
    # loop over lines
    : read_next_line
    n

    # flag with \x01<URL>\x02 all occurences of URLs
    s/"(http[^"]+)"/\x01\1\x02/g

    # we found at least one URL if there is \x01 in the pattern space
    /\x01/{

        # extract each occurence to the end of pattern  space with a newline
        : again
        s/^([^\x01]*)\x01([^\x02]*)\x02(.*)$/\1\3\n\2/
            t again

        # remove everything in front of separator - the unparsed part of line
        s/^[^\n]*\n//
        # add URLs to hold space
        H
    }

    # if this is the last line, we should finally print something!, and, exit
    ${
        # grab the hold space
        x
        # replace the separator for a ;
        s/\n/;/g
        # print and exit successfully
        p
        q 0
    }

    # here we go again!
    b read_next_line
}

'

对于以下输入：

Delivered-To: SOME@ADDRESS.COM
Received: by X.X.X.X with SMTP id SOMEID;
        Wed, 9 Oct 2019 01:55:58 -0700 (PDT)
X-Received: by X.X.X.X with SMTP id SOMEID;
        Wed, 09 Oct 2019 01:55:58 -0700 (PDT)
Return-Path: <SOME@ADDRESS.COM>
Received: from SOME.URL.COM (SOME.OTHER.URL.COM. [X.X.X.X])
        by SOME.THIRD.URL.COM with ESMTP id SOMEID
        for <SOME@ADDRESS.COM>;
        Wed, 09 Oct 2019 01:55:58 -0700 (PDT)

SOME_HTML
SOME_HTML
href="http://URL1"><img
SOME_HTML
src="http://URL2"
SOME_HTML
SOMEHTML src="http://URL3" SOMEHTML src="http://URL4"

输出：

Wed, 09 Oct 2019 01:55:58 -0700 (PDT);Received: from SOME.URL.COM (SOME.OTHER.URL.COM. [X.X.X.X]);http://URL1;http://URL2;http://URL3;http://URL4