NodeJS 如何从 aws s3 存储桶将文件下载到磁盘？

Question

我的目标：

显示一个对话框，提示用户保存从 aws 下载的文件。

我的问题：

我目前正在使用 awssum-amazon-s3 创建下载流。但是，我只设法将文件保存到我的服务器或将其流式传输到命令行...正如您从我的代码中看到的那样，我最后一次尝试是尝试手动设置失败的内容处置标头。我不能使用 res.download() 因为已经设置了标头？

我怎样才能实现我的目标？

我的节点代码：

app.post('/dls/:dlKey', function(req, res, next){
        // download the file via aws s3 here
        var dlKey = req.param('dlKey');

        Dl.findOne({key:dlKey}, function(err, dl){
            if (err) return next(err);
            var files = dl.dlFile;

            var options = {
                BucketName    : 'xxxx',
                ObjectName    : files,
            };

            s3.GetObject(options, { stream : true }, function(err, data) {
                // stream this file to stdout
                fmt.sep();
                data.Headers['Content-Disposition'] = 'attachment';
                console.log(data.Headers);
                data.Stream.pipe(fs.createWriteStream('test.pdf'));
                data.Stream.on('end', function() {
                    console.log('File Downloaded!');
                });
            });
        });

        res.end('Successful Download Post!');
    });

我的角度代码：

$scope.dlComplete = function (dl) {
        $scope.procDownload = true;
        $http({
            method: 'POST',
            url: '/dls/' + dl.dlKey
        }).success(function(data/*, status, headers, config*/) {
            console.log(data);
            $location.path('/#!/success');
        }).error(function(/*data, status, headers, config*/) {
            console.log('File download failed!');
        });
    };

此代码的目的是让用户使用生成的密钥下载文件一次。

Answer 1

这是在最新版本的 aws-sdk 上使用流式传输的整个代码

var express = require('express');
var app = express();
var fs = require('fs');

app.get('/', function(req, res, next){
    res.send('You did not say the magic word');
});


app.get('/s3Proxy', function(req, res, next){
    // download the file via aws s3 here
    var fileKey = req.query['fileKey'];

    console.log('Trying to download file', fileKey);
    var AWS = require('aws-sdk');
    AWS.config.update(
      {
        accessKeyId: "....",
        secretAccessKey: "...",
        region: 'ap-southeast-1'
      }
    );
    var s3 = new AWS.S3();
    var options = {
        Bucket    : '/bucket-url',
        Key    : fileKey,
    };

    res.attachment(fileKey);
    var fileStream = s3.getObject(options).createReadStream();
    fileStream.pipe(res);
});

var server = app.listen(3000, function () {
    var host = server.address().address;
    var port = server.address().port;
    console.log('S3 Proxy app listening at http://%s:%s', host, port);
});

Answer 2

此代码适用于我最新的库：

var s3 = new AWS.S3();
var s3Params = {
    Bucket: 'your bucket',
    Key: 'path/to/the/file.ext'
};
s3.getObject(s3Params, function(err, res) {
    if (err === null) {
       res.attachment('file.ext'); // or whatever your logic needs
       res.send(data.Body);
    } else {
       res.status(500).send(err);
    }
});

Answer 3

只需从 S3 和 WriteStream 创建一个 ReadStream 到您要下载的位置。找到下面的代码。非常适合我：

var AWS = require('aws-sdk');
var path = require('path');
var fs = require('fs');

AWS.config.loadFromPath(path.resolve(__dirname, 'config.json'));
AWS.config.update({
  accessKeyId: AWS.config.credentials.accessKeyId,
  secretAccessKey: AWS.config.credentials.secretAccessKey,
  region: AWS.config.region
});

var s3 = new AWS.S3();
var params = {
  Bucket: '<your-bucket>', 
  Key: '<path-to-your-file>'
};
let readStream = s3.getObject(params).createReadStream();
let writeStream = fs.createWriteStream(path.join(__dirname, 's3data.txt'));
readStream.pipe(writeStream);

Answer 4

您已经知道解决问题最重要的是什么：您可以将来自 S3 的文件流通过管道传输到任何可写流，无论是文件流……还是将发送到客户端的响应流！

s3.GetObject(options, { stream : true }, function(err, data) {
    res.attachment('test.pdf');
    data.Stream.pipe(res);
});

注意使用res.attachment 将设置正确的标题。您还可以查看 this answer 关于流和 S3 的信息。

Answer 5

为此，我使用React frontend 和node js backend。前端我使用 Axios。我用这个点击按钮下载文件。

==== Node js 后端代码 (AWS S3) ======

//在GET方法里面我调用了这个函数

    public download = (req: Request, res: Response) => {
    const keyName = req.query.keyName as string;
    if (!keyName) {
        throw new Error('key is undefined');
    }
    const downloadParams: AWS.S3.GetObjectRequest = {
        Bucket: this.BUCKET_NAME,
        Key: keyName
    };

    this.s3.getObject(downloadParams, (error, data) => {
        if (error) {
            return error;
        }
        res.send(data.Body);
        res.end();
    });
};

====== React js 前端代码 ========

//该函数处理下载按钮onClick

  const downloadHandler = async (keyName: string) => {
  const response = await axiosInstance.get( //here use axios interceptors
    `papers/paper/download?keyName=${keyName}`,{
      responseType:'blob', //very very important dont miss (if not downloaded file unsupported to view)
    }
  );
  const url = window.URL.createObjectURL(new Blob([response.data]));
  const link = document.createElement("a");
  link.href = url;
  link.setAttribute("download", "file.pdf"); //change "file.pdf" according to saved name you want, give extension according to filetype
  document.body.appendChild(link);
  link.click();
  link.remove();
};

------或者（如果你使用的是普通的 axios 而不是axios interceptors）-----

axios({
   url: 'http://localhost:5000/static/example.pdf',
   method: 'GET',
   responseType: 'blob', // very very important
}).then((response) => {
   const url = window.URL.createObjectURL(new Blob([response.data]));
   const link = document.createElement('a');
   link.href = url;
   link.setAttribute('download', 'file.pdf');
   document.body.appendChild(link);
   link.click();
});

更多请参考以下文章 1.article 1 2.article 2

Answer 6

使用 aws SDK v3

npm install @aws-sdk/client-s3

下载代码

import { GetObjectCommand } from "@aws-sdk/client-s3";
/**
 * download a file from AWS and send to your rest client
 */
app.get('/download', function(req, res, next){
    var fileKey = req.query['fileKey'];

    var bucketParams = {
        Bucket: 'my-bucket-name',
        Key: fileKey,
    };

    res.attachment(fileKey);
    var fileStream = await s3Client.send(new GetObjectCommand(bucketParams));
    // for TS you can add: if (fileStream.Body instanceof Readable)
    fileStream.Body.pipe(res)
});