慢速 S3 -> EC2 传输。错误的代码？

Question

我正在将此脚本与 PHP5-CLI 一起使用，以将文件从 Amazon S3 下载到 EC2 实例，但速度相当慢（160 MB 需要 1 分钟，大约 2 MB/s）。假设至少 10 MB/s 应该是可能的。我在代码中做错了吗？

require 'aws-sdk/sdk.class.php';

$s3 = new AmazonS3(array(
        'key' => '********',
        'secret' => '******************'
));
$s3->get_object('mahbucket', 'filename.tar.gz', array('fileDownload' => 'downloaded.tar.gz'));

桶在eu-west-1，实例在eu-west-1a

(Cross posted to AWS forums)

Answer 1

您的脚本看起来不错，但不久前我遇到了类似的问题，我无法解决......

我的新方法是使用s3cmd 在 S3 存储桶和我的 EC2 实例之间传输文件。您只需配置一次，然后就可以通过 PHP 触发它。它也更安全，因为您不必将 AWS 凭证存储在脚本中。

我认为速度还可以：

user@mothership:~/s3# s3cmd put test.bin s3://data.example.com/test.bin

test.bin -> s3://data.example.com/test.bin  [part 1 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.39 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 2 of 13, 15MB]
 15728640 of 15728640   100% in    0s    15.55 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 3 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.18 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 4 of 13, 15MB]
 15728640 of 15728640   100% in    0s    17.32 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 5 of 13, 15MB]
 15728640 of 15728640   100% in    0s    18.87 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 6 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.58 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 7 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.29 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 8 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.95 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 9 of 13, 15MB]
 15728640 of 15728640   100% in    0s    15.33 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 10 of 13, 15MB]
 15728640 of 15728640   100% in    0s    17.01 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 11 of 13, 15MB]
 15728640 of 15728640   100% in    0s    15.36 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 12 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.56 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 13 of 13, 10MB]
 11256320 of 11256320   100% in    0s    14.15 MB/s  done

user@mothership:~/s3# s3cmd get s3://data.example.com/test.bin test.bin.new
s3://data.example.com/test.bin -> test.bin.new  [1 of 1]
 200000000 of 200000000   100% in   11s    16.20 MB/s  done

也许这篇文章也很有趣：Network performance within Amazon EC2 and to Amazon S3