System.Net.Http.HttpClient 是否遭受 HttpWebRequest.AllowWriteStreamBuffering 的影响？答案

【问题标题】：Does System.Net.Http.HttpClient suffer from HttpWebRequest.AllowWriteStreamBuffering?System.Net.Http.HttpClient 是否遭受 HttpWebRequest.AllowWriteStreamBuffering 的影响？
【发布时间】：2013-04-16 12:36:35
【问题描述】：

我一直在尝试使用 System.Net.Http.HttpClient 发布一个更大的文件 (+1GB)，但它会引发 SystemOutOfMemory 异常：

at System.Net.ScatterGatherBuffers.AllocateMemoryChunk(Int32 newSize)
at System.Net.ScatterGatherBuffers..ctor(Int64 totalSize)
at System.Net.ConnectStream.EnableWriteBuffering()
at System.Net.HttpWebRequest.SetRequestSubmitDone(ConnectStream submitStream)
at System.Net.Connection.CompleteStartRequest(Boolean onSubmitThread, HttpWebRequest request, TriState needReConnect)
at System.Net.Connection.SubmitRequest(HttpWebRequest request, Boolean forcedsubmit)
at System.Net.ServicePoint.SubmitRequest(HttpWebRequest request, String connName)
at System.Net.HttpWebRequest.SubmitRequest(ServicePoint servicePoint)
at System.Net.HttpWebRequest.BeginGetRequestStream(AsyncCallback callback, Object state)
at System.Net.Http.HttpClientHandler.StartGettingRequestStream(RequestState state)
at System.Net.Http.HttpClientHandler.PrepareAndStartContentUpload(RequestState state)

显然，HttpWebRequest 出现了类似的问题，如下所述：http://support.microsoft.com/kb/908573。

有没有办法将底层网络请求的AllowWriteStreamBuffering设置为false？我找不到。

干杯，

【问题讨论】：

标签： .net httpclient out-of-memory

【解决方案1】：

为了节省其他感兴趣的人的时间，我正在回答我自己的问题。

经过几次测试，异常似乎与问题中讨论的HttpWebRequest 相同。我使用 Microsoft.AspNet.WebApi 版本 4.0.20710.0。

下面是两段等价的代码；前者在大文件上失败，而后者工作正常。

顺便说一句，尽管存在问题，HttpClient 的整体优势变得非常明显 :-)

使用`HttpClient`

var clientRef = new System.Net.Http.HttpClient(
    new HttpClientHandler()
    {
        Credentials = new System.Net.NetworkCredential(MyUsername, MyPassword)
    });
clientRef.BaseAddress = new Uri(serverAddress);
clientRef.DefaultRequestHeaders.ExpectContinue = false;
clientRef.PostAsync(
    MyFavoriteURL,
    new System.Net.Http.StreamContent(inputStream)).ContinueWith(
        requestTask =>
        {
            HttpResponseMessage response = requestTask.Result;
            response.EnsureSuccessStatusCode();
        }, TaskContinuationOptions.LongRunning).Wait();

使用`HttpWebRequest`

// Preauthenticate
var req  = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(MyFavoriteURL);
req.Credentials = new System.Net.NetworkCredential(MyUsername, MyPassword);
req.Method = "POST";
req.PreAuthenticate = true;
req.Timeout = 10000;
using (var resp = (System.Net.HttpWebResponse)req.GetResponse())
{
     if (resp.StatusCode != System.Net.HttpStatusCode.Accepted && resp.StatusCode != System.Net.HttpStatusCode.OK)
     {
         throw new Exception("Authentication error");
     }
}

// Upload
req = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(MyFavoriteURL);
req.Credentials = new System.Net.NetworkCredential(MyUsername, MyPassword);
req.Method = "POST";
req.PreAuthenticate = true;
req.Timeout = 1200000;
req.ContentLength = inputStream.Length;
req.ContentType = "application/binary";
req.AllowWriteStreamBuffering = false;
req.Headers.ExpectContinue = false;
using (var reqStream = req.GetRequestStream())
{
    inputStream.CopyTo(reqStream);
}

using (var resp = (System.Net.HttpWebResponse)req.GetResponse())
{
    if (resp.StatusCode != System.Net.HttpStatusCode.Accepted && resp.StatusCode != System.Net.HttpStatusCode.OK)
    {
        throw new Exception("Error uploading document");
    }
}

【讨论】：

为什么ExpectContinue 设置为false？ PostAsync 之后的 ContinueWith 块的用途是什么？

【解决方案2】：

我在上传大文件时也遇到了同样的问题。添加我的发现，以便对某人有所帮助。

HttpClient根据以下条件判断是否做缓冲，

if( HttpRequestMessage.Headers.TransferEncodingChunked == true ||     HttpRequestMessage.Content.Headers.ContentLength != null )
{
  //do streamed upload
}else
{
  //do buffered upload.
}

我已将 PushedStreamContent 用作 HttpRequestMessage.Content，因为我的服务器最多接受每个请求 150 MB。一旦我将 TransferEncodingChunked 设置为 True，内存峰值就会减少。

【讨论】：

【解决方案3】：

我认为你有一个重复的标题，你可以删除前者

req.Headers.Add("ExpectContinue", "false");
...
req.Headers.ExpectContinue = false;

【讨论】：

我删除了不需要的行。谢谢

【解决方案4】：

帮助我发送一个大文件，而无需通过“中继控制器”进行缓冲。
我开始工作的场景是：
Client web app(spa, etc) -> RelayController -> 存储http数据流的最终内容控制器

var request = new HttpRequestMessage()
{
  RequestUri = new Uri("https://someurl/api/upload"),
  Method = HttpMethod.Post,
};
request.Headers.TransferEncodingChunked = true;
request.Content = new StreamContent(Request.Body);

await httpClient.SendAsync(request);

花了很多时间让它工作，得到了 Prakash P 的领导，他发布了 httpclient 流与缓冲逻辑。

【讨论】：

使用HttpClient

使用HttpWebRequest

使用`HttpClient`

使用`HttpWebRequest`