如何从 HTTPS url 读取 html 源代码 [关闭]

Question

我正在尝试使用以下代码读取 c# 中 https url 的 html 源代码：

 WebClient webClient = new WebClient();
 string htmlString = w.DownloadString("https://www.targetUrl.com");

这对我不起作用，因为我得到了编码的 html 字符串。我尝试使用 HtmlAgilityPack 但没有任何帮助。

HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(htmlString);

Answer 1

该 URL 正在返回一个 gzip 压缩字符串。 WebClient 默认情况下不支持此功能，因此您需要转而使用底层的 HttpWebRequest 类。 feroze 在这里公然剽窃答案 - Automatically decompress gzip response via WebClient.DownloadData

class MyWebClient : WebClient
{
    protected override WebRequest GetWebRequest(Uri address)
    {
        HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
        request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
        return request;
    }
}

Answer 2

ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
WebClient webClient = new WebClient();
string htmlString = webClient.DownloadString(url);