使用异步在 Node JS 中一个接一个地下载视频？

Question

我想在一个系列中一个接一个地下载视频。

也就是说，第一个应该在第二个开始之前完全下载，第二个应该在第三个开始之前完全下载，依此类推。

我有以下目录结构-

video-downloader
├── index.js
├── videos.js
├── package.json

package.json

{
  "name": "video-downloader",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "download": "^7.1.0"
  },
  "scripts": {
    "start": "node index"
  }
}

video.js

const videos = [
  {
    url: 'https://video.com/lesson1.mp4',
    name: 'Lesson 1',
  },
  {
    url: 'https://video.com/lesson2.mp4',
    name: 'Lesson 2',
  },
  .
  .
  .
  {
    url: 'https://video.com/lesson2.mp4',
    name: 'Lesson 100',
  }
]

index.js

const fs = require('fs')
const download = require('download')

const videos = require('./videos')

const OUTPUT_DIR = 'Downloads'

fs.mkdir(OUTPUT_DIR, () => {
   main()
})

const main = () => {
    videos.map((video, i) => {
        console.log(`Downloaded file ${i + 1} of ${videos.length} (${video.name})`)
        download(video.url).pipe(
            fs.createWriteStream(`${OUTPUT_DIR}/${video.name}.mp4`),
        )
    })
}

这会并行地逐块下载视频。一次下载所有视频，但没有一个在另一个开始之前完成。

如何连续下载？

我知道我应该使用 http://caolan.github.io/async/ 之类的东西，但它需要一个函数签名并且我将 videos 作为一个数组，所以我不知道该怎么做。

Answer 1

您可以在标准for循环中使用await关键字，事情会按顺序处理，并等待每次下载再继续。

const fs = require('fs')
const download = require('download')
const videos = require('./videos')
const util = require('util')

const mkdirAsync = util.promisify(fs.mkdir)

const OUTPUT_DIR = 'Downloads'

const main = async () => {
  await mkdirAsync(OUTPUT_DIR)

  for (let i = 0; i < videos.length; i++) {
    const video = videos[i]
    const data = await download(video.url)
    fs.writeFileSync(`${OUTPUT_DIR}/${video.name}.mp4`, data)
    console.log(`Downloaded file ${i + 1} of ${videos.length} (${video.name})`)
  }
}

main()

Answer 2

你可以使用.reduce和promise来顺序解析，如下：

const fs = require('fs')
const sh = require('shelljs')
const download = require('download')

const videos = require('./videos')

const OUTPUT_DIR = 'Downloads'

sh.mkdir('-p', OUTPUT_DIR)

videos = videos.reduce((acc, item) => {

  return acc.then(() => {
    return new Promise((resolve) => {

      // Here you are using it as a Duplex Stream, not a promise,
      // therefore, you must check when the stream emits the 'end' event
      // so you can proceed further
      let stream = download(video.url)
        .pipe(fs.createWriteStream(`${OUTPUT_DIR}/${video.name}.mp4`));

      stream.on('end', () => {
        console.log(`stream done ${item}`);
        resolve(item);
      })

    })

  });

}, Promise.resolve());

// 'videos' is now a promise
videos.then((lastPromise) => {

  // using reduce will return the last evaluated item(promise)
  // but reaching the last one means the promises before that have been resolved

  console.log('all files were downloaded');


})

Answer 3

为此尝试异步等待。先下载再同步写入。

const fs = require('fs');
const sh = require('shelljs');
const download = require('download');

const videos = require('./videos');

const OUTPUT_DIR = 'Downloads';

sh.mkdir('-p', OUTPUT_DIR);

videos.forEach(async (video, i) => {
  console.log(`Downloading ${video.name}. Fil${i + 1}/${videos.length} - `);
  const data = await download(video.url);
  fs.writeFileSync(`${OUTPUT_DIR}/${video.name}.mp4`, data);
});