如何通过 bash 脚本向 condor 提交 wget 作业？

Question

我在向 condor 提交 wget 作业时遇到问题。 我可以使用 wget 通过命令行从 url 下载文件。

$ wget https://wordpress.org/latest.zip
--2019-01-24 16:43:42--  https://wordpress.org/latest.zip
Resolving wordpress.org... 198.143.164.252
Connecting to wordpress.org|198.143.164.252|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11383968 (11M) [application/zip]
Saving to: “latest.zip”

100%[======================================>] 11,383,968  --.-K/s   in 0.09s 

2019-01-24 16:43:43 (117 MB/s) - “latest.zip” saved [11383968/11383968]

但如果我将命令保存到 bash 脚本文件“test.sh”中，如下所示：

#!/bin/sh
wget https://wordpress.org/latest.zip

然后提交给condor：

#!/usr/bin/env condor_submit
Executable = test.sh
Universe = vanilla
output = tmp.out
error = tmp.error
Log = tmp.log
Queue 1

会出现“连接超时”的错误。

--2019-01-24 16:53:50--  https://wordpress.org/latest.zip
Resolving wordpress.org... 198.143.164.252
Connecting to wordpress.org|198.143.164.252|:443... failed: Connection timed out

但 test.sh 在命令行中运行良好，如下所示：

$./test.sh

我把“tesh.sh”改成：

#!/bin/sh
wget --debug --verbose https://wordpress.org/latest.zip

输出是：

Setting --verbose (verbose) to 1
DEBUG output created by Wget 1.12 on linux-gnu.

--2019-01-24 17:25:58--  https://wordpress.org/latest.zip
Resolving wordpress.org... 198.143.164.252
Caching wordpress.org => 198.143.164.252
Connecting to wordpress.org|198.143.164.252|:443... Closed fd 3
failed: Connection timed out.

Answer 1

我怀疑问题出在“https”上：SSL/TLS 握手失败。

如果您使用网络浏览器连接到 HTTPS 站点，则使用浏览器的信任库（通常，这与您桌面的 Windows 信任库是一回事）。

建议：

1. 从命令行验证您的“test.sh”是否有效。

2. 修改您的脚本以运行wget --debug --trace ...，并检查输出。

3. 如果只是为了故障排除，也可以试试--no-check-certificate。

4. 查看这里了解更多详情：

   GNU Wget 1.18 Manual

---

附录：

如果您运行wget --debug --verbose ...，您应该会看到如下内容：

$ wget --debug --verbose https://wordpress.org/latest.zip
Setting --verbose (verbose) to 1
Setting --verbose (verbose) to 1
DEBUG output created by Wget 1.19.4 on linux-gnu.

Reading HSTS entries from /home/xxxxx/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name 'latest.zip' (UTF-8) -> 'latest.zip' (UTF-8)
--2019-01-24 15:49:12--  https://wordpress.org/latest.zip
Resolving wordpress.org (wordpress.org)... 198.143.164.252
Caching wordpress.org => 198.143.164.252
Connecting to wordpress.org (wordpress.org)|198.143.164.252|:443... connected.
Created socket 3.
Releasing 0x000055afcdc64650 (new refcount 1).
Initiating SSL handshake.
Handshake successful; connected socket 3 to SSL handle 0x000055afcdc64750
certificate:
  subject: CN=*.wordpress.org,OU=Domain Control Validated
  issuer:  CN=Go Daddy Secure Certificate Authority - G2,OU=http://certs.godaddy.com/repository/,O=GoDaddy.com\\, Inc.,L=Scottsdale,ST=Arizona,C=US
X509 certificate successfully verified and matches host wordpress.org

---request begin---
GET /latest.zip HTTP/1.1
User-Agent: Wget/1.19.4 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: wordpress.org
Connection: Keep-Alive
...

如果您没有看到任何...我会联系您的网络管理员以了解可能阻止您的 Condor 应用程序的防火墙或代理。