grep，剪切并从文件中删除 \n

Question

我正在处理一个输入文件，该文件包含一个新行上的用户 ID 列表。在 bash 脚本中，我在该输入文件上运行一个 while 循环，使用 grep -E 执行 ldapsearch 查询来过滤我想要的结果。生成的输出文件目前格式如下（/mountpoint/out_file_1.out）；

uid=user_id1,cn=Users,ou=Department,dc=myORG    
LDAPresource=myORG_RESname1   LDAPresource=myORG_RESname2  
uid=user_id2,cn=Users,ou=Department,dc=myORG  
LDAPresource=myORG_RESname2   LDAPresource=myORG_RESname3

然而，所需的输出应该如下所示；

user_id1;myORG_RESname1
user_id1;myORG_RESname2
user_id2;myORG_RESname2
user_id2;myORG_RESname3

到目前为止，我已经尝试使用 grep 和 cut 来实现上述所需的输出。这是我在上面的第一个结果文件上运行的确切命令：

grep -E '(^uid=|myORG_RESname1|myORG_RESname2|myORG_RESname3)' /mountpoint/out_file_1.out | cut -d, -f1 >&5

导致第二个输出 (/mountpoint/out_file_2.out);

uid=user_id1  
LDAPresource=myORG_RESname1     
LDAPresource=myORG_RESname2  

再次，用 cut 运行另一个 grep：

grep -E 'LDAPresource|uid=' /mountpoint/out_file_2.out | cut -d= -f2 >&6

最终产生这个输出（/mountpoint/out_file_3.out）：

user_id1  
myORG_RESname1  
myORG_RESname2  

这“几乎”是我所需要的。我生成的最后一个输出需要去掉换行符并为每个已找到的资源名称重复用户 ID，如已针对所需输出 (/mountpoint/final_output.out) 描述的那样：

user_id1;myORG_RESname1  
user_id1;myORG_RESname2 

使用：

tr '\n' ';' < input_file > output_file 没有给我想要的结果...

任何想法如何实现这一目标？非常感谢任何帮助。

编辑：

这是我正在运行的实际 bash 脚本以供参考：

#!/bin/bash

# assign file descriptor for input fd
exec 3< /mountpoint/userlist
# assign file descriptor for output fd unfiltered
exec 4> /mountpoint/out_file_1.out
# assign file descriptor for output fd filtered
exec 5> /mountpoint/out_file_2.out
# assign file descriptor for output fd final
exec 6> /mountpoint/out_file_3.out

while IFS= read -ru 3 LINE; do
    ldapsearch -h IPADDR -D "uid=admin,cn=Users,ou=Department,dc=myDC" -w somepwd "(uid=$LINE)" LDAPresource >&4
    grep -E '(^uid=|Resource1|Resource2|Resource3)' /mountpoint/out_file_1.out | cut -d, -f1 >&5
    grep -E 'TAMresource|uid=' /mountpoint/out_file_2.out | cut -d= -f2 >&6
    #tr '\n' ';' < input_filename > file
done
# close fd #3 inputfile
exec 3<&-
# close fd #4 & 5 outputfiles
exec 4>&-
exec 5>&-
# exit with 0 success status
exit 0

Answer 1

使用您展示的示例，请尝试以下操作。使用 GNU awk 中的示例编写和测试。

awk '
match($0,/uid=[^,]*/){
  val1=substr($0,RSTART+4,RLENGTH-4)
  next
}
{
  val=""
  while($0){
    match($0,/LDAPresource=[^ ]*/)
    val=(val?val OFS:"")(val1 ";" substr($0,RSTART+13,RLENGTH-13))
    $0=substr($0,RSTART+RLENGTH)
  }
  print val
}' Input_file

说明：为上述添加详细说明。

awk '                                 ##Starting awk program from here.
match($0,/uid=[^,]*/){                ##Using match function to match regex uid= till comma comes in current line.
  val1=substr($0,RSTART+4,RLENGTH-4)  ##Creating val1 variable which has sub string of matched regex of above.
  next                                ##next will skip all further statements from here.
}
{
  val=""                              ##Nullifying val variable here.
  while($0){                          ##Running loop till current line value is not null.
    match($0,/LDAPresource=[^ ]*/)    ##using match to match regex from string LDAPresource= till space comes.
    val=(val?val OFS:"")(val1 ";" substr($0,RSTART+13,RLENGTH-13))  ##Creating val which has val1 ; and sub string of above matched regex.
    $0=substr($0,RSTART+RLENGTH)      ##Saving rest of line in current line.
  }
  print val                           ##Printing val here.
}' Input_file                         ##Mentioning Input_file name here.

Answer 2

您要执行的转换的规范不清楚。看来您希望成对处理行，采用每对第一行表示的 uid 属性和每对第二行指定的两个 LDAPresource 属性，并将它们组合成两行，每行包含一个 @ 987654322@对。

首先，我不会为此使用grep 或cut。 sed 或 awk 将是更合适的工具。我更喜欢sed 而不是awk，但我确信一个非常简单的awk 脚本可以一次性完成这项工作。对于sed，我会使用两个：

• 首先，从您的输入到您的第三个输出，如下所示：

  sed 's/^[^=]*=//; s/,.*//; n; s/LDAPresource=//g; s/ \{1,\}/\n/'
  

• 其次，将生成的三组线组合起来以实现您想要的输出：

  sed 's/$/;/; h; N; x; N; H; x; s/;\n/;/g'
  

您可以将它们组合成一个命令（尽管我当然建议为此编写一个脚本，而不是在命令行中全部输入）：

sed 's/^[^=]*=//; s/,.*//; n; s/LDAPresource=//g; s/ \{1,\}/\n/' /mountpoint/out_file_1.out |
  sed 's/$/;/; h; N; x; N; H; x; s/;\n/;/g'

说明

每个给定的sed 命令都指定了一个以分号分隔的步骤序列，这些步骤将在一个循环中执行，直到输入用完为止。

这是第一个多行格式，带有 cmets

# The next line of input is implicitly read into sed's pattern space, sans trailing newline

# Replace the leading substring up to the first '=' with nothing (that is, delete it)
s/^[^=]*=//

# Replace the substring from the first comma to the end of the line with nothing.
# This leaves just the uid value.
s/,.*//

# Print the contents of the pattern space followed by a newline (supposes that the
# -n command line option has not been given) and replace the contents of the pattern
# space with the next line of input.
n

# Replace all substrings 'LDAPresource=' in the pattern space with nothing
s/LDAPresource=//g

# Replace the first (and only) run of one or more consecutive space characters with a newline
s/ \{1,\}/\n/

# The remaining contents of the pattern space and a trailing newline are printed at this point
# (assuming no '-n' option) and the cycle repeats.

第二个是：

# The next line of input is implicitly read into sed's pattern space sans trailing newline

# Substitute a semicolon (;) for the zero-length space at the end of the line (that
# is, append a semicolon).
s/$/;/

# Copy the contents of the pattern space into the hold space.  Both spaces then contain
# the uid plus a semicolon
h

# Append a newline followed by the next line of input (sans trailing newline) to the
# pattern space
N

# Swap the contents of the pattern and hold spaces.
x

# Append a newline followed by the next line of input (sans trailing newline) to the
# pattern space
N

# Append a newline followed by the contents of the pattern space to the hold space.
# After this, the contents of the hold space have the form
# <uid>;<newline><resource1><newline><uid>;<newline><resource2>
H

# Swap the pattern and hold spaces
x

# Replace each (semicolon, newline) pair with just a semicolon.  This completes
# joining the uid and resource pairs into semicolon-(only-)delimited form,
# leaving a newline between each pair
s/;\n/;/g

# The remaining contents of the pattern space and a trailing newline are printed at this
# point (assuming no '-n' option) and the cycle repeats.

Answer 3

$ awk -F'[=,[:space:]]+' -v OFS=',' 'NR%2{uid=$2; next} {print uid, $2 ORS uid, $4}' file
user_id1,myORG_RESname1
user_id1,myORG_RESname2
user_id2,myORG_RESname2
user_id2,myORG_RESname3