如何同步不同的shell实例

Question

我想同步/“序列化”同时运行的特定 shell 的多个实例。

可选地，一个参数子集可以确定进程是可以立即运行还是必须等待。

要清楚：

如果我同时运行以下三个命令（在本例中schema 和table 选项确定锁定）：

loadTable --schema dev --table person --file mydata1.txt
loadTable --schema dev --table person --file mydata2.txt
loadTable --schema dev --table enterprise --file mydata3.txt

我想要：

• first和third命令可以同时运行
  因为部分参数不同（schema 和 table）
• second 命令必须等待 first 命令的结束
  因为部分参数是相同的（schema 和 table）

对我来说有 2 个解决方案：

• loadTable shell 必须自己管理等待队列。
• 父 代理 shell 必须管理它

我有一些想法，但似乎有点复杂......

感谢您的帮助

Answer 1

您可以将第二步简化和安全一点
（通过这种方式，即使 $pid_file 在上次执行期间被删除，它也可以工作）

# Second step : waiting eventual previous process
previous_pid=$(cat $pid_file)
previous_pid=${previous_pid:00} # 00 is an impossible pid
while [ -e /proc/${previous_pid} ]
do
    echo "Another process already running with process_id ${previous_pid}"        
    sleep $delay
done

Answer 2

我做了一个同步功能。

我现在可以简单地在我的脚本开头调用（在 args 解析之后）：

synchronize $myTable-$mySchema

它可以完成这项工作，但我们也许可以简化它。

代码：

function synchronize() {
    key=${1:-noKey};    
    shell_fullname=$(readlink -f $0)
    shell_basename=$(basename $shell_fullname)    
    hash=$(echo "${shell_fullname}-${key}" | md5sum | cut -b-32)
    delay=2
    pid_file=/tmp/${shell_basename}.${hash}.pid    
    current_pid=$$        

    echo "synchronize$shell_basename($key)"
    (    
        # First step : lock file         
        isUnLock=true
        echo "trying to acquire lock"
        while $isUnLock
        do
            # Wait for lock on file desciptor 200 for 10 seconds
            flock -x -w $delay 200 && isUnLock=false
            $isUnLock && echo "Waiting lock for"                   
        done

        # here : isUnLock must normally be false
        $isUnLock && echo "unable to acquire lock" # not possible for me
        $isUnLock && exit 255 # bad luck...
        $isUnLock || echo "lock OK"         

        # Second step : waiting eventual previous process
        while [ -e ${pid_file} ] && kill -0 `cat $pid_file` > /dev/null 2>&1
        do
            echo "Another process already running with process_id $(cat ${pid_file})"        
            sleep $delay
        done

        # here : previous shell stop and the current process has lock the pid_file
        echo $current_pid > ${pid_file}

        # now we can unlock the pid file and the current shell can be running alone safely
        # (just fail if somebody delete the pid_file...)        

    ) 200>$pid_file.lock # add suffix, flock seems make empty file after lock... 

}

key=$1 # construct key using args
synchronize$key

最轻的版本

function synchronize() {
    key=${1:-noKey};    
    shell_fullname=$(readlink -f $0)
    shell_basename=$(basename $shell_fullname)    
    hash=$(echo "${shell_fullname}-${key}" | md5sum | cut -b-32)
    delay=10
    pid_file=/tmp/${shell_basename}.${hash}.pid    
    current_pid=$$        

    echo "synchronize $shell_basename($key) pid_file=$pid_file"
    (    
        # First step : lock file, Wait for lock on file desciptor 200
        echo "trying to acquire lock"
        flock -x 200 
        echo "lock acquired"

        # Second step : waiting eventual previous process        
        [ -s $pid_file ] && previous_pid=$(cat $pid_file) || previous_pid=00 # 00 is an impossible pid        
        [ -e /proc/${previous_pid} ] && echo "Another process already running with process_id $previous_pid"        
        while [ -e /proc/${previous_pid} ]; do sleep $delay; done

        # here : previous shell is stop and the current process has lock the pid_file        
        echo $current_pid > ${pid_file}        
        echo "current pid $current_pid is running"

        # now we can unlock the pid file and the current shell can be running several minutes      

    ) 200>$pid_file.lock # add suffix, flock seems make empty file after lock... 

}

Answer 3

如果我猜对了，你需要这样做：

loadTable --schema dev --table person --file mydata1.txt &
loadTable --schema dev --table enterprise --file mydata3.txt &
wait %1 && loadTable --schema dev --table person --file mydata2.txt &
wait

在这种情况下，您将并行运行命令 1 和 3，然后等待第一个命令完成并运行第二个命令。之后，您等待所有这些都完成