C#在控制台应用程序上异步下载多个文件时遇到问题[重复]

Question

在你们对这是一个重复的问题大发雷霆之前，我已经花了两天时间研究这个问题，观看有关异步编程的 youtube 教程，浏览类似的 stackoverflow 帖子等，但我终生无法理解了解如何将文件的异步并行下载应用到我的项目中。

首先，一些背景知识：

我正在创建一个程序，当通过用户给出查询输入时，它将调用 twitch API 并下载剪辑。

我的程序分为两部分

1- 生成一个 .json 文件的网络爬虫，其中包含下载文件所需的所有详细信息和

2 - 下载器。

第 1 部分运行良好，生成 .json 文件没有问题。

我的下载器包含对 Data 类的引用，该类是常见属性和方法的处理程序，例如我的 ClientID、Authentication、OutputPath、JsonFile、QueryURL。它还包含为这些属性赋值的方法。

这是我的 FileDownloader.cs 的两个方法有问题：

public async static void DownloadAllFiles(Data clientData)
{
    data = clientData;

    data.OutputFolderExists();


    // Deserialize .json file and get ClipInfo list
    List<ClipInfo> clips = JsonConvert.DeserializeObject<List<ClipInfo>>(File.ReadAllText(data.JsonFile));
            
    tasks = new List<Task>();

    foreach(ClipInfo clip in clips)
    {
        tasks.Add(DownloadFilesAsync(clip));
    }

    await Task.WhenAll(tasks);
}

private async static Task DownloadFilesAsync(ClipInfo clip)
{
    WebClient client = new WebClient();
    string url = GetClipURL(clip);
    string filepath = data.OutputPath + clip.id + ".mp4";

    await client.DownloadFileTaskAsync(new Uri(url), filepath);
}

这只是我下载文件的众多尝试之一，我从这篇文章中得到了这个想法：

stackoverflow_link

我还尝试了 IAmTimCorey 的 YouTube 视频中的以下方法：

video_link

我已经花了很多个小时来解决这个问题，但老实说，我无法弄清楚为什么它在我的任何尝试中都不起作用。非常感谢您的帮助。

谢谢，

本

以下是我的全部代码，如果有人出于任何原因需要它。

代码结构：

我下载的唯一外部库是 Newtonsoft.Json

ClipInfo.cs

using System;
using System.Collections.Generic;
using System.Text;

namespace Downloader
{
    public class ClipInfo
    {
        public string id { get; set; }
        public string url { get; set; }
        public string embed_url { get; set; }
        public string broadcaster_id { get; set; }
        public string broadcaster_name { get; set; }
        public string creator_id { get; set; }
        public string creator_name { get; set; }
        public string video_id { get; set; }
        public string game_id { get; set; }
        public string language { get; set; }
        public string title { get; set; }
        public int view_count { get; set; }
        public DateTime created_at { get; set; }
        public string thumbnail_url { get; set; }
    }
}

Pagination.cs

namespace Downloader
{
    public class Pagination
    {
        public string cursor { get; set; }
    }

}

Root.cs

using System.Collections.Generic;

namespace Downloader
{
    public class Root
    {
        public List<ClipInfo> data { get; set; }
        public Pagination pagination { get; set; }
    }
}

Data.cs

using System;
using System.IO;

namespace Downloader
{
    public class Data
    {
        private static string directory = Directory.GetCurrentDirectory();
        private readonly static string defaultJsonFile = directory + @"\clips.json";
        private readonly static string defaultOutputPath = directory + @"\Clips\";
        private readonly static string clipsLink = "https://api.twitch.tv/helix/clips?";

        public string OutputPath { get; set; }
        public string JsonFile { get; set; }
        public string ClientID { get; private set; }
        public string Authentication { get; private set; }
        public string QueryURL { get; private set; }
    

        public Data()
        {
            OutputPath = defaultOutputPath;
            JsonFile = defaultJsonFile;
        }
        public Data(string clientID, string authentication)
        {
            ClientID = clientID;
            Authentication = authentication;
            OutputPath = defaultOutputPath;
            JsonFile = defaultJsonFile;
        }
        public Data(string clientID, string authentication, string outputPath)
        {
            ClientID = clientID;
            Authentication = authentication;
            OutputPath = directory + @"\" + outputPath + @"\";
            JsonFile = OutputPath + outputPath + ".json";
        }

        public void GetQuery()
        {
            Console.Write("Please enter your query: ");
            QueryURL = clipsLink + Console.ReadLine();
        }

        public void GetClientID()
        {
            Console.WriteLine("Enter your client ID");
            ClientID = Console.ReadLine();
        }

        public void GetAuthentication()
        {
            Console.WriteLine("Enter your Authentication");
            Authentication = Console.ReadLine();
        }

        public void OutputFolderExists()
        {
            if (!Directory.Exists(OutputPath))
            {
                Directory.CreateDirectory(OutputPath);
            }
        }

    }
}

JsonGenerator.cs

using System;
using System.IO;
using System.Net.Http.Headers;
using System.Net.Http;
using System.Threading.Tasks;
using Newtonsoft.Json;
using System.Linq;


namespace Downloader
{
    public static class JsonGenerator
    {
        // This class has no constructor.
        // You call the Generate methods, passing in all required data.
        // The file will then be generated.
        private static Data data;

        public static async Task Generate(Data clientData)
        {
            data = clientData;
            string responseContent = null;

            // Loop that runs until the api request goes through
            bool authError = true;
            while (authError)
            {
                authError = false;
                try
                {
                    responseContent = await GetHttpResponse();
                }
                catch (HttpRequestException)
                {
                    Console.WriteLine("Invalid authentication, please enter client-ID and authentication again!");
                    data.GetClientID();
                    data.GetAuthentication();

                    authError = true;
                }
                catch (Exception e)
                {
                    Console.WriteLine(e.Message);
                    authError = true;
                }
            }

            data.OutputFolderExists();
            GenerateJson(responseContent);
        }

        // Returns the contents of the resopnse to the api call as a string
        private static async Task<string> GetHttpResponse()
        {
            // Creating client
            HttpClient client = new HttpClient();

            if (data.QueryURL == null)
            {
                data.GetQuery();
            }


            // Setting up request
            HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Get, data.QueryURL);

            // Adding Headers to request
            requestMessage.Headers.Add("client-id", data.ClientID);
            requestMessage.Headers.Authorization = new AuthenticationHeaderValue("Bearer", data.Authentication);

            // Receiving response to the request
            HttpResponseMessage responseMessage = await client.SendAsync(requestMessage);

            // Gets the content of the response as a string
            string responseContent = await responseMessage.Content.ReadAsStringAsync();

            return responseContent;
        }

        // Generates or adds to the .json file that contains data on each clip
        private static void GenerateJson(string responseContent)
        {
            // Parses the data from the response to the api request
            Root responseResult = JsonConvert.DeserializeObject<Root>(responseContent);

            // If the file doesn't exist, we need to create it and add a '[' at the start
            if (!File.Exists(data.JsonFile))
            {
                FileStream file = File.Create(data.JsonFile);
                file.Close();
                // The array of json objects needs to be wrapped inside []
                File.AppendAllText(data.JsonFile, "[\n");
            }
            else
            {
                // For a pre-existing .json file, The last object won't have a comma at the
                // end of it so we need to add it now, before we add more objects
                string[] jsonLines = File.ReadAllLines(data.JsonFile);
                File.WriteAllLines(data.JsonFile, jsonLines.Take(jsonLines.Length - 1).ToArray());
                File.AppendAllText(data.JsonFile, ",");
            }

            // If the file already exists, but there was no [ at the start for whatever reason,
            // we need to add it
            if (File.ReadAllText(data.JsonFile).Length == 0 || File.ReadAllText(data.JsonFile)[0] != '[')
            {
                File.WriteAllText(data.JsonFile, "[\n" + File.ReadAllText(data.JsonFile));
            }

            string json;

            // Loops through each ClipInfo object that the api returned
            for (int i = 0; i < responseResult.data.Count; i++)
            {
                // Serializes the ClipInfo object into a json style string
                json = JsonConvert.SerializeObject(responseResult.data[i]);

                // Adds the serialized contents of ClipInfo to the .json file
                File.AppendAllText(data.JsonFile, json);

                if (i != responseResult.data.Count - 1)
                {
                    // All objects except the last require a comma at the end of the
                    // object in order to correctly format the array of json objects
                    File.AppendAllText(data.JsonFile, ",");
                }

                // Adds new line after object entry
                File.AppendAllText(data.JsonFile, "\n");
            }
            // Adds the ] at the end of the file to close off the json objects array
            File.AppendAllText(data.JsonFile, "]");
        }
    }
}

FileDownloader.cs

using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Threading.Tasks;

namespace Downloader
{
    public class FileDownloader
    {
        private static Data data;
        private static List<Task> tasks;
        public async static void DownloadAllFiles(Data clientData)
        {
            data = clientData;

            data.OutputFolderExists();


            // Deserialize .json file and get ClipInfo list
            List<ClipInfo> clips = JsonConvert.DeserializeObject<List<ClipInfo>>(File.ReadAllText(data.JsonFile));

            tasks = new List<Task>();

            foreach (ClipInfo clip in clips)
            {
                tasks.Add(DownloadFilesAsync(clip));
            }

            await Task.WhenAll(tasks);
        }

        private static void GetData()
        {
            if (data.ClientID == null)
            {
                data.GetClientID();
            }
            if (data.Authentication == null)
            {
                data.GetAuthentication();
            }
            if (data.QueryURL == null)
            {
                data.GetQuery();
            }
        }

        private static string GetClipURL(ClipInfo clip)
        {
            // Example thumbnail URL:
            // https://clips-media-assets2.twitch.tv/AT-cm%7C902106752-preview-480x272.jpg
            // You can get the URL of the location of clip.mp4
            // by removing the -preview.... from the thumbnail url */

            string url = clip.thumbnail_url;
            url = url.Substring(0, url.IndexOf("-preview")) + ".mp4";
            return url;
        }
            
        private async static Task DownloadFilesAsync(ClipInfo clip)
        {
            WebClient client = new WebClient();
            string url = GetClipURL(clip);
            string filepath = data.OutputPath + clip.id + ".mp4";

            await client.DownloadFileTaskAsync(new Uri(url), filepath);
        }

        private static void FileDownloadComplete(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
        {
            tasks.Remove((Task)sender);
        }
    }
}

Program.cs

using System;
using System.Threading.Tasks;
using Downloader;

namespace ClipDownloader
{
    class Program
    {
        private static string clientID = "{your_client_id}";
        private static string authentication = "{your_authentication}";
        async static Task Main(string[] args)
        {
            Console.WriteLine("Enter your output path");
            string outputPath = Console.ReadLine();


            Data data = new Data(clientID, authentication, outputPath);
            Console.WriteLine(data.OutputPath);

            //await JsonGenerator.Generate(data);
            FileDownloader.DownloadAllFiles(data);
        }
    }
}

我通常输入的示例查询是“game_id=510218”

Answer 1

async void 是你的问题

改变

public static async void DownloadAllFiles(Data clientData)

到

public static async Task DownloadAllFiles(Data clientData)

那你就等着吧

await FileDownloader.DownloadAllFiles(data);

---

更长的故事：

async void 在未观察到的情况下运行（即发即弃）。你不能等待他们完成。本质上，一旦您的程序启动任务，它就会完成并拆除 App Domain 和所有子任务，让您相信没有任何工作。

Answer 2

我试图在这里尽可能地停留在主题上，但是当使用 JsonConvert.DeserializeObject{T} 时，T 不应该是一个封装的根对象类型吗？我从来没有像你那样使用它，所以我只是好奇这是否可能是你的错误。我可能完全错了，如果我错了，请放过我，但 JSON 是基于键：值的。直接反序列化为 List 并没有什么意义。除非反序列化器中有特殊情况？ List 将是一个文件，它是一个纯粹的 ClipInfo 值数组，被反序列化为 List{T} 的成员（private T[] _items、private int _size 等）它需要一个父根对象。

// current JSON file format implication(which i dont think is valid JSON?(correct me please) 
clips:
[
  // clip 1
  {  "id": "", "url": "" },

  // clip N
  {  "id": "", "url": "" },
]

// correct(?) JSON file format
{ // { } is the outer encasing object
    clips:
    [
        // clip 1
        {  "id": "", "url": "" },

        // clip N
        {  "id": "", "url": "" },
    ]
}

class ClipInfoJSONFile
{
    public List<ClipInfo> Info { get; set; }
}

var clipInfoList = JsonConverter.DeserializeObject<ClipInfoJSONFile>(...);