使用 awk 拆分文本文件

Question

示例文本文件将是这样的

ID   Z4WTH3_9ACTN            Unreviewed;       182 AA.
AC   Z4WTH3; A0SD0SDF;
AC   Z12SDFG3; ADFFGDF;
DT   11-JUN-2014, integrated into UniProtKB/TrEMBL.
SQ   SEQUENCE   182 AA;  20675 MW;  B85D18AC3B1F0E75 CRC64;
     MNFLEYNKDE KLHFNYKKSC GLWLIVVALI IFAATVIGGK QIINMSVFSF GYVAAFLSIN
//
ID   Z4WXU8_9ACTN            Unreviewed;       203 AA.
AC   Z4WXU8;
AC   QWERDFV1;
DT   11-JUN-2014, integrated into UniProtKB/TrEMBL.
SQ   SEQUENCE   203 AA;  23224 MW;  35F1AE4342F6B3AC CRC64;
     MDCKSIRSEV LWQVVRLREK LMNFLEYNKD EKLCFNYKKS CGLWLIVVAL IIFAATVIGG
//
ID   Z9JHX1_9GAMM            Unreviewed;       132 AA.
AC   Z9JHX1;
SQ   SEQUENCE   132 AA;  13880 MW;  0E09988C0F3ED155 CRC64;
     MKISVDTNVL ARAVLQDDAN QGRSASTLLK DASLIAVSLP CLCELVWILS RGAKLSKEDV
//

实际文件为 100GB 文件 该文件仅包含一个“ID”行，并且始终以“ID”行开头。以“//”结尾

“AC”行可以是多行。我们必须将第一个“AC”行的第一个元素作为文件名。

需要根据“//”将该文件拆分为多个文件。 每个文件都应命名为以 AC 开头的行中的文本。

所以输出文件看起来像

Z4WTH3.txt

ID   Z4WTH3_9ACTN            Unreviewed;       182 AA.
AC   Z4WTH3; A0SD0SDF;
AC   Z12SDFG3; ADFFGDF;
DT   11-JUN-2014, integrated into UniProtKB/TrEMBL.
SQ   SEQUENCE   182 AA;  20675 MW;  B85D18AC3B1F0E75 CRC64;
     MNFLEYNKDE KLHFNYKKSC GLWLIVVALI IFAATVIGGK QIINMSVFSF GYVAAFLSIN
//

Z4WXU8.txt

ID   Z4WXU8_9ACTN            Unreviewed;       203 AA.
AC   Z4WXU8;
AC   QWERDFV1;
DT   11-JUN-2014, integrated into UniProtKB/TrEMBL.
SQ   SEQUENCE   203 AA;  23224 MW;  35F1AE4342F6B3AC CRC64;
     MDCKSIRSEV LWQVVRLREK LMNFLEYNKD EKLCFNYKKS CGLWLIVVAL IIFAATVIGG
//

Z9JHX1.txt

ID   Z9JHX1_9GAMM            Unreviewed;       132 AA.
AC   Z9JHX1;
SQ   SEQUENCE   132 AA;  13880 MW;  0E09988C0F3ED155 CRC64;
     MKISVDTNVL ARAVLQDDAN QGRSASTLLK DASLIAVSLP CLCELVWILS RGAKLSKEDV
//

Answer 1

关注awk 可能对您有所帮助。

awk '/^ID/{close(filename);val=$2;sub(/_.*/,"",val);filename=val".txt"} {print > filename}'  Input_file

解决方案二： 根据 OP 文件名应该来自字符串 AC 所以现在也添加以下解决方案。

awk '/^ID/{close(filename);first=$0 ORS;next} /^AC/{val=$2;sub(";","",val);filename=val".txt";print first $0 > filename;next} {print > filename}'  Input_file

或者，如果 Input_file 在所有部分中都没有ID 标签，那么我们可以在AC 标签中编写close 函数，如下所示：

awk '/^ID/{first=$0 ORS;next} /^AC/{close(filename);val=$2;sub(";","",val);filename=val".txt";print first $0 > filename;next} {print > filename}'  Input_file

说明：现在也添加解决方案说明：

awk '
/^ID/{                       ##Searching string ID here if it is present in any line then do following:
  first=$0 ORS;              ##Creating variable named first whose value is current line with ORS(output record separator).
  next}                      ##next is awk default keyword which will sip further statements.
/^AC/{                       ##Checking here condition if a line contains string AC then do following:
  close(filename);           ##Closing the file which was previously written heer so that we will NOT get too many open files issues.
  val=$2;                    ##Creating variable named val and keeping its value as 2nd field of current line.
  sub(";","",val);           ##Using sub utility of awk to subsitute semi colon with NULL in variable val here.
  filename=val".txt";        ##Creating variable named filename whose value is variable val and .txt(creating output file names here).
  print first $0 > filename; ##Printing variable first and current line in the output file here.
  next                       ##next will skip all further statements now.
}
{
  print > filename           ##Printing the current lines into output file whoever are NOT satisfying the above 2 conditions.
}
'  Input_file                ##Mentioning the Input_file name here.

Answer 2

另一个使用RS（GNU awk 由于 multichar RS）来分隔记录：

$ gawk '
BEGIN {
    RS=ORS="\n//\n"          # record separators
}
{
    for(i=1;i<=NF;i++)       # go thru each field in record
        if($i=="AC") {       # once AC found
            f=$(i+1) "TXT"   # next one is the filename
            sub(/;/,".",f)   # replace ; with .
            print > f        # print to file (multiple AC:s lead to multiple files)
            close(f)         # close to avoid problem with too many open files
                             # overwrites files when files with same name
        }
}' file

文件：

$ ls -l Z*
-rw-r--r-- 1 james james 254 Feb 27 09:23 Z4WTH3.TXT
-rw-r--r-- 1 james james 254 Feb 27 09:23 Z4WXU8.TXT
-rw-r--r-- 1 james james 202 Feb 27 09:23 Z9JHX1.TXT

文件内部：

$ cat Z9JHX1.TXT
ID   Z9JHX1_9GAMM            Unreviewed;       132 AA.
AC   Z9JHX1;
SQ   SEQUENCE   132 AA;  13880 MW;  0E09988C0F3ED155 CRC64;
     MKISVDTNVL ARAVLQDDAN QGRSASTLLK DASLIAVSLP CLCELVWILS RGAKLSKEDV
//

Answer 3

使用 GNU awk 进行多字符 RS 和 RT：

awk -v RS='\n//\n' -v ORS= -F'[[:space:];]+' '{print $0 RT > ($7".txt")}' file

使用任何 awk：

awk -F'[[:space:];]+' '
    $1 == "AC" { out = $2".txt" }
    { rec = rec $0 ORS }
    $0 == "//" {
        printf "%s", rec > out
        close out
        rec = ""
    }
' file