【问题标题】:In a CSV file, subtotal 2 columns based on a third one, using AWK in KSH在 CSV 文件中,基于第三列小计 2 列,在 KSH 中使用 AWK
【发布时间】:2018-04-17 18:18:02
【问题描述】:

免责声明:

    1) English is my second language, so please forgive any grammatical horrors you may find. I am pretty confident you will be able to understand what I need despite these.
    2) I have found several examples in this site that address questions/problems similar to mine, though I was unfortunately not able to figure out the modifications that would need to be introduced to fit my needs.

“问题”:

我有一个如下所示的 CSV 文件:

c1,c2,c3,c4,c5,134.6,,c8,c9,SERVER1,c11
c1,c2,c3,c4,c5,0,,c8,c9,SERVER1,c11
c1,c2,c3,c4,c5,0.18,,c8,c9,SERVER2,c11
c1,c2,c3,c4,c5,0,,c8,c9,SERVER2,c11
c1,c2,c3,c4,c5,416.09,,c8,c9,SERVER3,c11
c1,c2,c3,c4,c5,0,,c8,c9,SERVER3,c11
c1,c2,c3,c4,c5,12.1,,c8,c9,SERVER3,c11
c1,c2,c3,c4,c5,480.64,,c8,c9,SERVER4,c11
c1,c2,c3,c4,c5,,83.65,c8,c9,SERVER5,c11
c1,c2,c3,c4,c5,,253.15,c8,c9,SERVER6,c11
c1,c2,c3,c4,c5,,18.84,c8,c9,SERVER7,c11
c1,c2,c3,c4,c5,,8.12,c8,c9,SERVER7,c11
c1,c2,c3,c4,c5,,22.45,c8,c9,SERVER7,c11
c1,c2,c3,c4,c5,,117.81,c8,c9,SERVER8,c11
c1,c2,c3,c4,c5,,96.34,c8,c9,SERVER9,c11

补充事实:

    1) File has 11 columns.
    2) The data in columns 1, 2, 3, 4, 5, 8, 9 and 11 is irrelevant in this case. In other words, I will only work with columns 6, 7 and 10.
    3) Column 10 will be typically alphanumeric strings (server names), though it may contain also "-" and/or "_".
    4) Columns 6 and 7 will have exclusively numbers, with up to two decimal places (A possible value is 0). Only one of the two will have data per line, never both.

我需要什么作为输出:

    - A single occurrence of every string in column 10 (as column 1), then the sum (subtotal) of it's values in column 6 (as column 2) and last, the sum (subtotal) of it's values in column 7 (as column 3).
    - If the total for a field is "0" the field must be left empty, but still must exist (it's respective comma has to be printed).
    - **Note** that the strings in column 10 will be already alphabetically sorted, so there is no need to do that part of the processing with AWK.

输出样本,使用上面的样本作为输入:

SERVER1,134.6,,
SERVER2,0.18,,
SERVER3,428.19,,
SERVER4,480.64,,
SERVER5,,83.65
SERVER6,,253.15
SERVER7,,26.96

我已经在这些页面中发现不是一个,而是两个 AWK oneliners,它们部分地完成了它所需要的:

awk -F "," 'NR==1{last=$10; sum=0;}{if (last != $10) {print last "," sum; last=$10; sum=0;} sum += $6;}END{print last "," sum;}' inputfile


awk -F, '{a[$10]+=$6;}END{for(i in a)print i","a[i];}' inputfile

我在这两种情况下的“问题”是相同的:

    - Subtotals of 0 are printed.
    - I can only handle the sum of one column at a time. Whenever I try to add the second one, I get either a syntax error or it does simply not print the third column at all.

提前感谢您的支持! 问候, 马丁

【问题讨论】:

    标签: awk ksh


    【解决方案1】:

    这样的?

    $ awk 'BEGIN{FS=OFS=","} 
                {s6[$10]+=$6; s7[$10]+=$7} 
             END{for(k in s6) print k,(s6[k]?s6[k]:""),(s7[k]?s7[k]:"")}' file | sort
    
    SERVER1,134.6,
    SERVER2,0.18,
    SERVER3,428.19,
    SERVER4,480.64,
    SERVER5,,83.65
    SERVER6,,253.15
    SERVER7,,49.41
    SERVER8,,117.81
    SERVER9,,96.34
    

    请注意,您对逗号的处理不一致,当最后一个字段为零时,您将添加一个额外的逗号(计算逗号)

    【讨论】:

    • Karafka:你对不一致的逗号是正确的。请注意 - 由于我的代码无法正常工作 - 我必须手动将所需的输出放在一起,因此会出现错误。顺便说一句,你的代码工作得像个笨蛋!: cat input.csv | awk '开始{FS=OFS=","} ; {s6[$10]+=$6; s7[$10]+=$7} ; END{for(k in s6) print k,(s6[k]?s6[k]:""),(s7[k]?s7[k]:"")}' SERVER1,134.6, SERVER2,0.18, SERVER3,428.19, SERVER4,480.64, SERVER5,,83.65 SERVER6,,253.15 SERVER7,,49.41 SERVER8,,117.81 SERVER9,,96.34 非常非常感谢。我已经把你的答案标记为正确的了!
    • 不客气。请注意,您不需要预先使用catawk 也可以打开文件。您不需要选择第一个答案,而是选择您认为是解决问题的最佳方法。从长远来看,这是一种更好的方法。这个的优点是不需要排序输入。缺点是事后需要整理。如果您没有可排序的键并且想要保持输入文件中的顺序,那么其他解决方案可能会更好。
    【解决方案2】:

    您发布的预期输出似乎与您发布的示例输入不匹配,所以我们猜测,但这可能是您正在寻找的:

    $ cat tst.awk
    BEGIN { FS=OFS="," }
    $10 != prev {
        if (NR > 1) {
            print prev, sum6, sum7
        }
        sum6 = sum7 = ""
        prev = $10
    }
    $6  { sum6 += $6 }
    $7  { sum7 += $7 }
    END { print prev, sum6, sum7 }
    
    $ awk -f tst.awk file
    SERVER1,134.6,
    SERVER2,0.18,
    SERVER3,428.19,
    SERVER4,480.64,
    SERVER5,,83.65
    SERVER6,,253.15
    SERVER7,,49.41
    SERVER8,,117.81
    SERVER9,,96.34
    

    【讨论】:

    • Ed:关于输出与输入不匹配的说法是正确的。请注意 - 由于我的代码无法正常工作 - 我必须手动将所需的输出放在一起,因此出现错误。顺便说一句,你的代码工作得像个笨蛋!太:猫输入.csv | awk '开始 {FS=OFS=","} ; $10 != prev {if (NR > 1) {print prev, sum6, sum7} ; sum6 = sum7 = "" ;上一页 = $10} ; $6 {sum6 += $6} ; $7 {sum7 += $7} ; END {print prev, sum6, sum7}' SERVER1,134.6, SERVER2,0.18, SERVER3,428.19, SERVER4,480.64, SERVER5,,83.65 SERVER6,,253.15 SERVER7,,49.41 SERVER8,,117.81 SERVER9,,96.34 非常感谢,非常男人!
    • 我将 Karakfa 的(也在工作的)回复标记为正确的回复,因为 - 因为他之前回复过 - 这似乎是最公平的做法。我也想标记你的回复,但这样做似乎取消了我选择的 karafka 回复的标记。如果有办法我可以同时标记两者,请告诉我,我一定会做的!再次感谢您花时间解决这个问题;)
    • 没问题。两者之间的主要区别在于,我的将始终按照服务器在输入中出现的顺序打印您的输出,而 karakfas 将根据服务器名称的字母顺序重新排列行。在输入的末尾添加 SERVER10 行,然后尝试两种解决方案以了解我的意思(SERVER10 将出现在我的 SERVER9 之后,但在 SERVER1 和 SERVER2 之间,使用 karakfas)。此外,如果您的文件变得非常大,然后您想使用我的文件,因为我的文件只在内存中存储了 2 个 sum 整数,而 karakfas 保留了由两个 sum 的所有服务器名称索引的数组。
    猜你喜欢
    • 2017-02-19
    • 1970-01-01
    • 2018-10-23
    • 2014-05-24
    • 2014-07-11
    • 2016-01-16
    • 1970-01-01
    • 1970-01-01
    • 2017-04-28
    相关资源
    最近更新 更多