如何通过多次过滤或搜索提取数据并获取多行使用 bash 脚本答案

【问题标题】：how to extract data with multiple filtering or searching and get multiple line use bash script如何通过多次过滤或搜索提取数据并获取多行使用 bash 脚本
【发布时间】：2021-05-27 10:53:57
【问题描述】：

我有这样的文件日志格式

输入

2021-02-21 12:12:32 Plugin: 

Table Number: 2001K

Operation: ProcessOpenTablet

Stored Procedure: sp_HDS_OpenTablet

Payload: {"id":"POS026","staff_id":"2","table_no":"2001K","customer_first_name":null}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-21 12:12:32 Plugin: 

Table Number: 2001K

Operation: GetListDishFormPOST

Stored Procedure: sp_HDS_GetDishesList

Payload: {"id":"POS026"}

Response: []

--------------------------------------------------------------------------------------------------------------------------------------------------  
2021-02-21 12:12:40 Plugin: 

Table Number: 2001K

Operation: SettlePayment

Stored Procedure: sp_HDS_Payment

Payload: {"id":"POS026","operator_no":"2","table_no":"2001K"}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-22 12:18:17 Plugin: 

Table Number: 2002K

Operation: ProcessOpenTablet

Stored Procedure: sp_HDS_OpenTablet

Payload: {"id":"POS027","staff_id":"2","table_no":"2001K","customer_first_name":null}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-22 12:18:18 Plugin: 

Table Number: 2002K

Operation: GetListDishFormPOST

Stored Procedure: sp_HDS_GetDishesList

Payload: {"id":"POS027"}

Response: []

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-22 12:18:28 Plugin: 

Table Number: 2002K

Operation: SettlePayment

Stored Procedure: sp_HDS_Payment

Payload: {"id":"POS027","operator_no":"2","table_no":"2001K"}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-22 15:18:28 Plugin: 

Table Number: 2002K

Operation: SettlePayment

Stored Procedure: sp_HDS_Payment

Payload: {"id":"POS028","operator_no":"2","table_no":"2001K"}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-22 12:24:04 Plugin: 

Table Number: 2003K

Operation: ProcessOpenTablet

Stored Procedure: sp_HDS_OpenTablet

Payload: {"id":"POS028","staff_id":"2","table_no":"2001K","customer_first_name":null}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-22 12:24:04 Plugin: 

Table Number: 2003K

Operation: GetListDishFormPOST

Stored Procedure: sp_HDS_GetDishesList

Payload: {"id":"POS026"}

Response: []

--------------------------------------------------------------------------------------------------------------------------------------------------

我想提取这个数据看下面的输出，可以动态搜索或过滤，例如过滤日期“2021-02-22”存储过程“sp_HDS_Payment”，结果为

预期输出

2021-02-22 12:18:28 Plugin: 

Table Number: 2002K

Operation: SettlePayment

Stored Procedure: sp_HDS_Payment

Payload: {"id":"POS027","operator_no":"2","table_no":"2001K"}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-22 15:18:28 Plugin: 

Table Number: 2002K

Operation: SettlePayment

Stored Procedure: sp_HDS_Payment

Payload: {"id":"POS028","operator_no":"2","table_no":"2001K"}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------

我试图使用 grep 或 cat 命令来做到这一点。但数据仅限于 1 行，我想要从“日期”（2021-02-22）到“响应”获取的数据示例。我该如何解决这个问题？

注意：过滤/搜索必须与两个条件匹配，在 DATE 输入 AND 存储过程 = sp_HDS_Payment

【问题讨论】：

空白行是文件的一部分吗？
我看到样本输入中有 3 个部分与 2012-02-22 和 sp_HDS_Payment 匹配，但只有 2 个部分在预期输出中...您是否遗漏了过滤器？还是应该所有 3 个部分都在预期的输出中？你总是有 2 个过滤器要搜索吗？过滤器的格式是什么？每个过滤器是否必须匹配一个专门标记的行（例如，如果在带有“存储过程”标签的行上，则只匹配sp_HDS_Payment）？
嗨@markp-fuso 很抱歉，我刚刚更新了示例数据日志。请再看一遍上面:)，谢谢你的指正。
嗨@Cyrus 是的，当然
@markp-fuso 是的，过滤器必须始终匹配 2 个条件，即日期和存储过程 = sp_HDS_Payment

标签： shell awk grep cut

【解决方案1】：

尝试以下 awk 解决方案：

 awk -v dat="2021-02-22" -v sp="sp_HDS_Payment" 'BEGIN { RS="--------------------------------------------------------------------------------------------------------------------------------------------------" } $0~dat && $0~sp { ORS=RS;print }' logfile

将记录分隔符设置为破折号，然后仅当记录具有指定的日期（dat）和存储过程（sp）时才打印记录

记录分隔符确定文件将如何拆分为单独的记录。 $0 ~ .... 然后检查记录 ($0) 是否包含使用 -v 传递到 awk 的 dat 或传递的 sp。

【讨论】：

海@Raman Sailopal，谢谢它的工作！但是你能在你的代码中一步一步地解释吗？尤其是在 BEGIN { RS="--------------------------" } $0~dat && $0~sp { ORS=RS;print }' ??
别担心，我已经添加了一些额外的解释

【解决方案2】：

您可以将此gnu-awk 与自定义RS 一起使用：

awk -v RS='-{5,}\n' -v dt='^2021-02-22 ' -v sp='sp_HDS_Payment' '{ORS=RT} $0 ~ dt && $0 ~ "\nStored Procedure: " sp "\n"' log.txt

2021-02-22 12:18:28 Plugin:

Table Number: 2002K

Operation: SettlePayment

Stored Procedure: sp_HDS_Payment

Payload: {"id":"POS027","operator_no":"2","table_no":"2001K"}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-22 15:18:28 Plugin:

Table Number: 2002K

Operation: SettlePayment

Stored Procedure: sp_HDS_Payment

Payload: {"id":"POS028","operator_no":"2","table_no":"2001K"}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-22 12:18:28 Plugin:

Table Number: 2002K

Operation: SettlePayment

Stored Procedure: sp_HDS_Payment

Payload: {"id":"POS027","operator_no":"2","table_no":"2001K"}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------
2021-02-22 15:18:28 Plugin:

Table Number: 2002K

Operation: SettlePayment

Stored Procedure: sp_HDS_Payment

Payload: {"id":"POS028","operator_no":"2","table_no":"2001K"}

Response: [{"ErrCode":"01","ErrMsg":""}]

--------------------------------------------------------------------------------------------------------------------------------------------------

展开形式：

awk -v RS='-{5,}\n' -v dt='^2021-02-22 ' -v sp='sp_HDS_Payment' '
   {ORS=RT}
   $0 ~ dt && $0 ~ "\nStored Procedure: " sp "\n"
' log.txt

【讨论】：