远程检查linux中是否所有文件都存在？

Question

我有一个下面的脚本，它迭代所有机器并远程检查每台机器：

• 特定目录是否存在？
• 该目录中是否存在所有 2000 个文件。

下面是代码：

for machine in "${MACHINES[@]}"; do
   dircheck=($(ssh -o "StrictHostKeyChecking no" user@${machine} [[ ! -d "$dir3" ]] \&\& exit 1 \; ls -t1 "$dir3"))
   if [[ $? != 0 ]] ;then
       echo "Folder $dir3 doesn't exist on $machine" >&2
       exit 1
   fi

    # this is for checking all the 2000 files
    # this check is very slow when it checks all 2000 files
    for el in ${FILES[@]}
    do
        printf -v cmd '%q ' test -e "$dir3/process_${el}.log"
        ssh user@${machine} "$cmd" \
        || { echo "File number $el missing on $machine." >&2;
             exit 1; }
    done
done

现在的问题是检查所有 2000 个文件需要很多时间，所以想看看是否有任何方法我们仍然可以做同样的事情，但速度有点快？

更新：

所以总的来说我的脚本是这样的：

readonly MACHINES=(machineA machineB machineC)
readonly dir3=/some_path
echo $dir3
FILES=({0..1999})

checkFunc() {
  test -d "$dir3" || echo "NODIR"

  local filename successCount=0
  while IFS= read -r filename; do
    test -e "$dir3/process_${filename}.log" && (( ++successCount ))
  done
  printf '%s\n' "$successCount"
}

for machine in "${MACHINES[@]}"; do
    actual=$(
      printf '%s\0' "${FILES[@]}" | \
        ssh "$machine" "$(declare -p dir3; declare -f checkFunc); checkFunc"
    ) || { echo "ERROR: Unable to retrieve remote file count" >&2; exit 1; }

    case $actual in
      (${#FILES[@]}) echo "SUCCESS: Expected, and found, $numberOfActuallyRemoteFiles files" ;;
      (NODIR)        echo "FAILURE: Directory $dir3 does not exist" ;;
      (*)            echo "FAILURE: Out of ${#FILES[@]} files, only $actual exist" ;;
    esac
done

Answer 1

与其创建 2,000 个单独的 SSH 连接，不如将整个循环推送到远程端：

# shell function which reads a NUL-delimited list of filenames on stdin
# and returns the number of them that actually exist on stdout, or "NODIR"
checkFunc() {
  test -d "$dir3" || { echo "NODIR"; return; }

  local filename successCount=0
  while IFS= read -r -d '' filename; do
    test -e "$dir3/process_${filename}.log" && (( ++successCount ))
  done
  printf '%s\n' "$successCount"
}

actual=$(
  printf '%s\0' "${FILES[@]}" | \
    ssh "$machine" "$(declare -p dir3; declare -f checkFunc); checkFunc"
) || { echo "ERROR: Unable to retrieve remote file count" >&2; exit 1; }

case $actual in
  (${#FILES[@]}) echo "SUCCESS: Expected, and found, $numberOfActuallyRemoteFiles files" ;;
  (NODIR)        echo "FAILURE: Directory $dir3 does not exist" ;;
  (*)            echo "FAILURE: Out of ${#FILES[@]} files, only $actual exist" ;;
esac

请注意，declare -p dir3 和 declare -f checkFunc 发出的字符串在 bash 执行时将分别定义 dir3 变量或 checkFunc 函数。