如何使用 NodeJS (NestJS) 将可恢复上传到 Google Cloud Storage (GCS)？

Question

我在将 NestJS 中的可恢复上传服务创建到 GCS 时遇到问题。

场景是，客户端从前端上传一个文件，在后端直接发送到GCS，而不是临时存储在BE服务器上。

这是我正在使用的代码 sn-p。

try {
  const filePath = path.join(directory, nameWithExtension);
  const file = this.bucket.file(filePath);
  const passthroughStream = new stream.PassThrough();
  passthroughStream.write(image.buffer);
  passthroughStream.end();

  const streamFileUpload = async () => {
    passthroughStream
      .pipe(file.createWriteStream({ resumable: true, gzip: true, public: true }))
      .on('finish', () => console.log(`resumable upload succeed`));
    return filePath;
  };

  const res = await streamFileUpload().catch((error) => {
    throw new Error(`${logPrefix} Error uploading ${filePath} ${error.message}`);
  });
  return `${process.env.GOOGLE_STORAGE_ENDPOINT}/${this.bucket.name}/${res}`;
} catch (error) {
  throw new Error(`${logPrefix} Error uploading ${error.message}`);
}

在 createWriteStream 上我包含了选项 resumable: true，但它似乎没有按预期工作。

我对这个 https://cloud.google.com/storage/docs/performing-resumable-uploads 很好奇，但还是不太明白。

非常感谢任何建议，谢谢！

2021 年 10 月 6 日更新

我把代码改成如下：

async resumableUpload(directory: string, image: MultipartFile, nameWithExtension: string): Promise<string> {
  const logPrefix = 'GoogleStorageService.resumableUpload:';
  const filePath = path.join(directory, nameWithExtension);
  const { buffer } = image;
  const blob = this.bucket.file(filePath);
  const promiseUpload = new Promise((resolve, reject) => {
    const blobStream = blob.createWriteStream({
      resumable: true,
      gzip: true,
      public: true,
    });
    blobStream
      .on('error', () => {
        reject(`${logPrefix} Unable to upload image, something went wrong`);
      })
      .on('finish', async () => {
        const publicUrl = new URL(process.env.GOOGLE_STORAGE_ENDPOINT || '');
        publicUrl.pathname = path.join(this.bucket.name, filePath);
        resolve(publicUrl.toString());
      })
      .end(buffer);
  });
  const response = promiseUpload
    .then((res: string) => res)
    .catch((err: Error) => {
      throw new Error(`${logPrefix} Error uploading ${err.message}`);
    });
  return response;
}

而且效果很好。

但如果有任何更好的方法，请不要犹豫，为这个问题提供最好的建议。谢谢

Answer 1

将@Wisnu 解决方案发布为社区 wiki，以获得更好的可见性。 Wisnu编辑以下代码后，可恢复上传服务开始工作。

async resumableUpload(directory: string, image: MultipartFile, nameWithExtension: string): Promise<string> {
  const logPrefix = 'GoogleStorageService.resumableUpload:';
  const filePath = path.join(directory, nameWithExtension);
  const { buffer } = image;
  const blob = this.bucket.file(filePath);
  const promiseUpload = new Promise((resolve, reject) => {
    const blobStream = blob.createWriteStream({
      resumable: true,
      gzip: true,
      public: true,
    });
    blobStream
      .on('error', () => {
        reject(`${logPrefix} Unable to upload image, something went wrong`);
      })
      .on('finish', async () => {
        const publicUrl = new URL(process.env.GOOGLE_STORAGE_ENDPOINT || '');
        publicUrl.pathname = path.join(this.bucket.name, filePath);
        resolve(publicUrl.toString());
      })
      .end(buffer);
  });
  const response = promiseUpload
    .then((res: string) => res)
    .catch((err: Error) => {
      throw new Error(`${logPrefix} Error uploading ${err.message}`);
    });
  return response;
}