awk 同一行值的几何平均值

Question

我有以下输入，如果“Cpd_number”和“ID3”相同，我想做几何平均。这些文件有很多数据，所以我们可能需要数组来完成这些技巧。但是，作为 awk 初学者，我不太确定如何开始。有人可以提供一些提示吗？

输入：

“ID1”,“Cpd_number”, “ID2”,”ID3”,”activity”
“95”,“123”,”4”,”5”,”10”
“95”, “123”,”4”,”5”,”100”
“95”, “123”,”4”,”5”,”1”
“95”, “123”,”4”,”6”,”10”
“95”, “123”,”4”,”6”,”100”
“95”, “456”,”4”,”6”,”10”
“95”, “456”,”4”,”6”,”100”

“95”、“123”、“4”、“5”三行应该做几何平均

两行“95”、“123”、“4”、“6”应该做几何平均

两行“95”、“456”、“4”、“6”应该做几何平均

这是所需的输出：

“ID1”,“Cpd_number”, “ID2”,”ID3”,”activity”
“95”,“123”,”4”,”5”,”10”
“95”, “123”,”4”,”6”,”31.62”
“95”, “456”,”4”,”6”,”31.62”

关于几何平均数的一些信息：

http://en.wikipedia.org/wiki/Geometric_mean

此脚本计算几何平均值

 #!/usr/bin/awk -f
 {
   b  = $1;   # value of 1st column
   C += log(b);  
   D++; 
 }

 END {
   print "Geometric mean  : ",exp(C/D);
   }

Answer 1

拥有这个文件：

$ cat infile
"ID1","Cpd_number","ID2","ID3","activity"
"95","123","4","5","10"
"95","123","4","5","100"
"95","123","4","5","1"
"95","123","4","6","10"
"95","123","4","6","100"
"95","456","4","6","10"
"95","456","4","6","100"

这件作品：

awk -F\" 'BEGIN{print}            # Print headers
      last != $4""$8 && last{     # ONLY When last key  "Cpd_number + ID3" 
          print line,exp(C/D)     # differs from actual , print line + average
          C=D=0}                  # reset acumulators
      { # This block process each line of infile
       C += log($(NF-1)+0)        # C calc
       D++                        # D counter
       $(NF-1)=""                 # Get rid of activity col ir order to print line
       line=$0                    # Line will be actual line without activity
       last=$4""$8}               # Store the key in orther to track switching 
      END{ # This block triggers after the complete file read
           # to print the last average that cannot be trigger during
           # the previous block 
          print line,exp(C/D)}' infile

会抛出：

 ID1 , Cpd_number ,  ID2 , ID3 ,   0
 95 ,  123 , 4 , 5 ,   10
 95 ,  123 , 4 , 6 ,   31.6228
 95 ,  456 , 4 , 6 ,   31.6228

还有一些工作要做格式化。

NOTE: char " is used  instead of “ and ”

EDIT：NF 是 file 中的字段数，因此 NF-1 将是倒数第二个：

$ awk -F\" 'BEGIN{getline}{print $(NF-1)}' infile                                                                                 
10
100
1
10
100
10
100

所以在：log($(NF-1)+0)我们对那个值应用对数函数（加0总和以确保数值）

D++ 只是一个计数器。

Answer 2

为什么要使用 awk，只需在 bash 中使用，使用 bc 或 calc 来处理浮点数学。您可以在http://www.isthe.com/chongo/src/calc/ 下载 calc（2.12.4.13-11 是最新的）。有可用的 rpm、二进制和源代码 tarball。在我看来，它远远优于bc。例程相当简单。 您需要从数据文件中删除多余的 " 引号，首先留下一个 csv 文件。这有帮助。请参阅下面的 cmets 中使用的 sed 命令。请注意，下面的几何平均值是 (id1*cpd*id2*id3) 的 4 次根。如果您需要不同的平均值，只需调整以下代码：

#!/bin/bash

##
##  You must strip all quotes from data before processing, or write more code to do
##  it here. Just do "$ sed -d 's/\"//g' < datafile > newdatafile" Then use 
##  newdatafile as command line argument to this program
##
##  Additionally, this script uses 'calc' for floating point math. go download it
##  from: http://www.isthe.com/chongo/src/calc/ (2.12.4.13-11 is latest). You can also
##  use bc if you like, but why, calc is so much better.
##

## test to make sure file passed as argument is readable
test -r "$1" || { echo "error: invalid input, usage: ${0//*\//} filename"; exit 1; }

## function to strip extraneous whitespace from input
trimWS() {
    [[ -z $1 ]] && return 1
    strln="${#1}"
    [[ strln -lt 2 ]] && return 1
    trimSTR=$1
    trimSTR="${trimSTR#"${trimSTR%%[![:space:]]*}"}"  # remove leading whitespace characters
    trimSTR="${trimSTR%"${trimSTR##*[![:space:]]}"}"  # remove trailing whitespace characters
    echo $trimSTR
    return 0
}

let cnt=0
let oldsum=0    # holds value to compare against new Cpd_number & ID3
product=1       # initialize product to 1
pcnt=0          # initialize the number of values in product
IFS=$',\n'      # Internal Field Separator, set to break on ',' or newline

while read newid1 newcpd newid2 newid3 newact || test -n "$act"; do

    cpd=`trimWS $cpd`  # trimWS from cpd (only one that needed it)

    # if first iteration, just output first row
    test "$cnt" -eq 0 && echo " $newid1 $newcpd $newid2 $newid3 $newact"

    # after first iteration, test oldsum -ne sum, if so do geometric mean
    # and reset product and counters
    if test "$cnt" -gt 0 ; then

        sum=$((newcpd+newid3))   # calculate sum to test against oldsum
        if test "$oldsum" -ne "$sum" && test "$cnt" -gt 1; then
            # geometric mean (nth root of product)
            # mean=`calc -p "root ($product, $pcnt)"`  # using calc
            mean=`echo "scale=6; e( l($product) / $pcnt)" | bc -l` # using bc
            echo " $id1 $cpd $id2 $id3  average: $mean"
            pcnt=0
            product=1
        fi

        # update last values to new values
        oldsum=$sum
        id1="$newid1"
        cpd="$newcpd"
        id2="$newid2"
        id3="$newid3"
        act="$newact"

        ((product*=act))  # accumulate product
        ((pcnt+=1))
    fi

    ((cnt+=1))

done < "$1"

输出：

# output using calc
ID1 Cpd_number  ID2 ID3 activity
95 123 4 5  average: 10
95 123 4 6  average: 31.62277660168379331999
95 456 4 6  average: 31.62277660168379331999

# output using bc
ID1 Cpd_number  ID2 ID3 activity
95 123 4 5  average: 9.999999
95 123 4 6  average: 31.622756
95 456 4 6  average: 31.622756

更新后的脚本计算正确的平均值。由于必须保留旧/新值来测试 cpd 和 id3 的变化，因此涉及更多。这可能是 awk 更简单的方法。但如果您以后需要更大的灵活性，bash 可能就是答案。