【问题标题】:.NET: Is it possible to get HttpWebRequest to automatically decompress gzip'd responses?.NET:是否可以让 HttpWebRequest 自动解压缩 gzip 的响应?
【发布时间】:2015-09-07 20:04:04
【问题描述】:

this answer 中,我描述了我如何在 HttpWebResponse 中的响应流周围包装 GZipStream 以解压缩它。

相关代码如下:

HttpWebRequest hwr = (HttpWebRequest) WebRequest.Create(url);
hwr.CookieContainer =
    PersistentCookies.GetCookieContainerForUrl(url);
hwr.Accept = "text/xml, */*";
hwr.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip, deflate");
hwr.Headers.Add(HttpRequestHeader.AcceptLanguage, "en-us");
hwr.UserAgent = "My special app";
hwr.KeepAlive = true;

using (var resp = (HttpWebResponse) hwr.GetResponse()) 
{
    using(Stream s = resp.GetResponseStream())
    {
        Stream s2 = s;
        if (resp.ContentEncoding.ToLower().Contains("gzip"))
            s2 = new GZipStream(s2, CompressionMode.Decompress);
        else if (resp.ContentEncoding.ToLower().Contains("deflate"))
            s2 = new DeflateStream(s2, CompressionMode.Decompress);

         ... use s2 ...
    }
}

有没有办法让 HttpWebResponse 自动提供解压缩流?换句话说,我可以从上面的代码中消除以下内容:

      Stream s2 = s;
      if (resp.ContentEncoding.ToLower().Contains("gzip"))
          s2 = new GZipStream(s2, CompressionMode.Decompress);
      else if (resp.ContentEncoding.ToLower().Contains("deflate"))
          s2 = new DeflateStream(s2, CompressionMode.Decompress);

谢谢。

【问题讨论】:

    标签: .net httpwebrequest gzip


    【解决方案1】:

    使用HttpWebRequest.AutomaticDecompression 属性如下:

    HttpWebRequest hwr = (HttpWebRequest) WebRequest.Create(url);
    hwr.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
    

    不需要手动添加Accept-Encoding HTTP 头;使用该属性时会自动添加。

    (另外,我知道这只是示例代码,但 HttpWebResponse 对象应放置在 using 块中,以便在您使用完毕后正确处理。)

    【讨论】:

    • 查清!谢谢。我怎么会错过呢?
    • “不需要手动添加 Accept-Encoding HTTP 标头” - 此外,如果您这样做,标头中的“gzip”和“deflate”值将重复,这可能会导致某些服务器拒绝请求。
    【解决方案2】:

    我的经验是它已经自动完成了。我必须通过将 Request 对象的 AutomaticDecompression 属性设置为 DecompressionMethods.None

    来明确禁用它

    【讨论】:

    • 嗯,这很奇怪。我需要重新运行我的测试。我以为我看到了压缩数据。我可能错了。谢谢。
    • 刚刚测试过 - 在我的代码中它没有自动开启。我必须阅读文档才能找到答案,但是。 . .当我明确启用它时,它消除了我做解压缩流的需要。谢谢你的回答。
    • 我很难找到一个可以按要求压缩我的数据的网站——google.com 不会这样做。 ——我想了一下,我输入了这个确切的 StackOverflow 问题的 URL,它起作用了——当我把它设置为 DecompressionMethods.None(.NET 4.5.1 中的明显默认值)时,我得到了 gzipped 字节。不错。
    • 根据你所拥有的框架版本不同,默认也不同
    【解决方案3】:

    我不认为 AutomaticDecompression 添加到 Header AcceptDecoding=gzip,deflate。 下面是我仅添加 AutomaticDecompression 时的请求和响应示例(没有显式添加 AcceptEncoding 标头)。可以看到,Header中根本没有AcceptEncoding

    //REQUEST 
    //===============================================================
    -       req {System.Net.HttpWebRequest} System.Net.HttpWebRequest
    +       base    {System.Net.HttpWebRequest} System.Net.WebRequest {System.Net.HttpWebRequest}
            _AuthInfo   null    System.Net.ICredentials
            _ContentLength  -1  long
    +       _HttpRequestHeaders {User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS .NET CF Web Services Client Protocol 3.5.7283.0)
    Authorization: Basic RElOT0I6MTExMQ==
    
    }   System.Net.WebHeaderCollection
            _MaximumAllowedRedirections 50  int
            _MediaType  null    string
    +       _OriginUri  {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
            _OriginVerb "GET"   string
    +       _Proxy  {System.Net.GlobalProxySelection.SystemWebProxy}    System.Net.IWebProxy {System.Net.GlobalProxySelection.SystemWebProxy}
            _ProxyAuthenticationState   null    System.Net.AuthenticationState
            _ReadWriteTimeout   300000  int
            _ServerAuthenticationState  null    System.Net.AuthenticationState
            _Timeout    100000  int
    +       _Timer  {System.Threading.Timer}    System.Threading.Timer
    +       _Uri    {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
            Aborted false   bool
            Accept  Could not evaluate expression   string
    +       Address {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
            AllowAutoRedirect   false   bool
            AllowWriteStreamBuffering   true    bool
            AutomaticDecompression  GZip | Deflate  System.Net.DecompressionMethods
            ChallengedUri   Could not evaluate expression   System.Uri
    +       ClientCertificates  {System.Security.Cryptography.X509Certificates.X509CertificateCollection}   System.Security.Cryptography.X509Certificates.X509CertificateCollection
            ConnectHostAndPort  "192.168.0.106:8084"    string
            Connection  Could not evaluate expression   string
            ConnectionGroupName Could not evaluate expression   string
            ContentLength   -1  long
            ContentType Could not evaluate expression   string
            ContinueDelegate    Could not evaluate expression   System.Net.HttpContinueDelegate
            Credentials Could not evaluate expression   System.Net.ICredentials
    +       CurrentAuthenticationState  {System.Net.AuthenticationState}    System.Net.AuthenticationState
            Expect  Could not evaluate expression   string
            hasEntityData   false   bool
            HaveResponse    false   bool
    +       Headers {User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS .NET CF Web Services Client Protocol 3.5.7283.0)
    Authorization: Basic RElOT0I6MTExMQ==
    
    }   System.Net.WebHeaderCollection
    +       IfModifiedSince {9/7/15 5:47:12 AM} System.DateTime
            KeepAlive   true    bool
            m_Aborted   false   bool
            m_allowAutoRedirection  false   bool
            m_allowWriteStreamBuffering true    bool
            m_AutomaticDecompression    GZip | Deflate  System.Net.DecompressionMethods
            m_boundConnection   null    System.Net.Connection
    +       m_clientCertificates    {System.Security.Cryptography.X509Certificates.X509CertificateCollection}   System.Security.Cryptography.X509Certificates.X509CertificateCollection
            m_connection    null    System.Net.Connection
            m_ConnectionClosedStopSendingEntityData false   bool
            m_connectionGroupName   null    string
            m_connectionUsers   0   int
            m_connMgrConnection -1  int
            m_continueDelegate  null    System.Net.HttpContinueDelegate
    +       m_continueFunction  {System.Threading.TimerCallback}    System.Threading.TimerCallback
    +       m_continueTimer null    System.Threading.Timer
            m_doneSendingEvent  null    System.Threading.ManualResetEvent
            m_errorResponse null    System.Net.WebException
            m_expectContinue    false   bool
            m_finishedWriting   false   bool
            m_httpWriteMode None    System.Net.HttpWriteMode
            m_IsCurrentAuthenticationStateProxy false   bool
            m_isSubmitting  false   bool
            m_KeepAlive true    bool
            m_numberRedirections    0   int
            m_Pipelined true    bool
            m_PreAuthenticate   false   bool
    +       m_readWriteTimer    {System.Threading.Timer}    System.Threading.Timer
            m_requestGeneration 0   int
    +       m_requestSentEvent  {System.Threading.ManualResetEvent} System.Threading.ManualResetEvent
    +       m_requestStream null    System.Net.HttpWriteStream
            m_requestStreamRetrieved    false   bool
            m_RequestSubmitted  false   bool
            m_response  null    System.Net.HttpWebResponse
            m_responseComplete  false   bool
    +       m_responseEvent {System.Threading.ManualResetEvent} System.Threading.ManualResetEvent
            m_responseRetrieved false   bool
            m_sentHeaders   false   bool
    +       m_setProxy  {System.Net.GlobalProxySelection.SystemWebProxy}    System.Net.IWebProxy {System.Net.GlobalProxySelection.SystemWebProxy}
    +       m_srvPoint  null    System.Net.ServicePoint
            m_startedReceiving  false   bool
            m_timedOut  0   int
    +       m_version   {1.1}   System.Version
            MaximumAutomaticRedirections    50  int
            MediaType   Could not evaluate expression   string
            Method  "GET"   string
            NtlmKeepAlive   false   bool
            Pipelined   true    bool
            PreAuthenticate false   bool
    +       ProtocolVersion {1.1}   System.Version
    +       Proxy   {System.Net.GlobalProxySelection.SystemWebProxy}    System.Net.IWebProxy {System.Net.GlobalProxySelection.SystemWebProxy}
            ProxyAuthenticatedConnectionSharing false   bool
    +       ProxyAuthenticationState    {System.Net.AuthenticationState}    System.Net.AuthenticationState
            ReadWriteTimeout    300000  int
            Referer Could not evaluate expression   string
    +       RequestUri  {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
    +       ResponseStatusCode  'req.ResponseStatusCode' threw an exception of type 'System.NullReferenceException' System.Net.HttpStatusCode {System.NullReferenceException}
            SendChunked false   bool
    +       ServerAuthenticationState   {System.Net.AuthenticationState}    System.Net.AuthenticationState
            ServicePoint    Could not evaluate expression   System.Net.ServicePoint
            SyncObject  {object}    object
            Timeout 100000  int
            TransferEncoding    Could not evaluate expression   string
            UserAgent   "Mozilla/4.0 (compatible; MSIE 6.0; MS .NET CF Web Services Client Protocol 3.5.7283.0)"    string
    +       UsesProxySemantics  'req.UsesProxySemantics' threw an exception of type 'System.NullReferenceException' bool {System.NullReferenceException}
    +       Static members  
    
    
    //RESPONSE 
    =====================================================
    -       res {System.Net.HttpWebResponse}    System.Net.HttpWebResponse
    +       base    {System.Net.HttpWebResponse}    System.Net.WebResponse {System.Net.HttpWebResponse}
            CharacterSet    ""  string
            ContentEncoding ""  string
            ContentLength   -1  long
            ContentType "text/xml"  string
    -       Headers {Date: Mon, 07 Sep 2015 11:49:36 GMT
    Server: Apache/2.2.29 (Win32)
    Vary: Accept-Encoding,User-Agent
    Content-Encoding: 
    Content-Length: 1481
    Keep-Alive: timeout=60, max=100
    Connection: Keep-Alive
    Content-Type: text/xml
    
    }   System.Net.WebHeaderCollection
    +       base    {Date: Mon, 07 Sep 2015 11:49:36 GMT
    Server: Apache/2.2.29 (Win32)
    Vary: Accept-Encoding,User-Agent
    Content-Encoding: 
    Content-Length: 1481
    Keep-Alive: timeout=60, max=100
    Connection: Keep-Alive
    Content-Type: text/xml
    
    }   System.Collections.Specialized.NameValueCollection {System.Net.WebHeaderCollection}
            m_IsHttpWebHeaderObject true    bool
    +       Static members      
    +       LastModified    {9/7/15 5:49:45 AM} System.DateTime
            m_contentLength -1  long
            m_decompressionMethod   GZip | Deflate  System.Net.DecompressionMethods
    -       m_httpResponseHeaders   {Date: Mon, 07 Sep 2015 11:49:36 GMT
    Server: Apache/2.2.29 (Win32)
    Vary: Accept-Encoding,User-Agent
    Content-Encoding: 
    Content-Length: 1481
    Keep-Alive: timeout=60, max=100
    Connection: Keep-Alive
    Content-Type: text/xml
    
    }   System.Net.WebHeaderCollection
    -       base    {Date: Mon, 07 Sep 2015 11:49:36 GMT
    Server: Apache/2.2.29 (Win32)
    Vary: Accept-Encoding,User-Agent
    Content-Encoding: 
    Content-Length: 1481
    Keep-Alive: timeout=60, max=100
    Connection: Keep-Alive
    Content-Type: text/xml
    
    }   System.Collections.Specialized.NameValueCollection {System.Net.WebHeaderCollection}
    +       base    {Date: Mon, 07 Sep 2015 11:49:36 GMT
    Server: Apache/2.2.29 (Win32)
    Vary: Accept-Encoding,User-Agent
    Content-Encoding: 
    Content-Length: 1481
    Keep-Alive: timeout=60, max=100
    Connection: Keep-Alive
    Content-Type: text/xml
    
    }   System.Collections.Specialized.NameObjectCollectionBase {System.Net.WebHeaderCollection}
            _allKeys    null    string[]
    +       AllKeys {string[8]} string[]
            m_IsHttpWebHeaderObject true    bool
    +       Static members      
            m_mediaType null    string
            m_method    "POST"  string
    -       m_responseStream    {System.Net.GZipWrapperStream}  System.IO.Stream {System.Net.GZipWrapperStream}
    +       [System.Net.GZipWrapperStream]  {System.Net.GZipWrapperStream}  System.Net.GZipWrapperStream
            base    {System.Net.GZipWrapperStream}  System.MarshalByRefObject {System.Net.GZipWrapperStream}
            _closeRecursionCounter  0   int
            CanRead true    bool
            CanSeek false   bool
            CanTimeout  false   bool
            CanWrite    false   bool
    +       Length  'res.m_responseStream.Length' threw an exception of type 'System.NotSupportedException' long {System.NotSupportedException}
    +       Position    'res.m_responseStream.Position' threw an exception of type 'System.NotSupportedException'   long {System.NotSupportedException}
    +       ReadTimeout 'res.m_responseStream.ReadTimeout' threw an exception of type 'System.InvalidOperationException'    int {System.InvalidOperationException}
    +       WriteTimeout    'res.m_responseStream.WriteTimeout' threw an exception of type 'System.InvalidOperationException'   int {System.InvalidOperationException}
    +       Static members      
            m_statusCode    200 int
            m_statusDescription "OK"    string
    +       m_url   {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
    +       m_version   {1.1}   System.Version
            Method  "POST"  string
    -       ProtocolVersion {1.1}   System.Version
            _Build  -1  int
            _Major  1   int
            _Minor  1   int
            _Revision   -1  int
            Build   -1  int
            Major   1   int
            Minor   1   int
            Revision    -1  int
    -       ResponseUri {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
            AbsolutePath    "/MyTestApp/Soap/IMyApp"    string
            AbsoluteUri "http://192.168.0.106:8084/MyTestApp/Soap/IMyApp"   string
            Authority   "192.168.0.106:8084"    string
            DnsSafeHost "192.168.0.106" string
            Fragment    ""  string
            HasAuthority    true    bool
            Host    "192.168.0.106" string
            HostNameType    IPv4    System.UriHostNameType
            HostType    IPv4HostType    System.Uri.Flags
            IsAbsoluteUri   true    bool
            IsDefaultPort   false   bool
            IsDosPath   false   bool
            IsFile  false   bool
            IsImplicitFile  false   bool
            IsLoopback  false   bool
            IsNotAbsoluteUri    false   bool
            IsUnc   false   bool
            IsUncOrDosPath  false   bool
            IsUncPath   false   bool
            LocalPath   "/MyTestApp/Soap/IMyApp"    string
            m_Flags IPv4HostType | AuthorityFound | NotDefaultPort | MinimalUriInfoSet | AllUriInfoSet  System.Uri.Flags
    +       m_Info  {System.Uri.UriInfo}    System.Uri.UriInfo
            m_OrigFileString    null    string
            m_String    "http://192.168.0.106:8084/MyTestApp/Soap/IMyApp"   string
    +       m_Syntax    {System.UriParser.BuiltInUriParser} System.UriParser {System.UriParser.BuiltInUriParser}
            OriginalString  "http://192.168.0.106:8084/MyTestApp/Soap/IMyApp"   string
            PathAndQuery    "/MyTestApp/Soap/IMyApp"    string
            Port    8084    int
            PrivateAbsolutePath "/MyTestApp/Soap/IMyApp"    string
            Query   ""  string
            Scheme  "http"  string
            SecuredPathIndex    0   ushort
    +       Segments    {string[4]} string[]
    +       Syntax  {System.UriParser.BuiltInUriParser} System.UriParser {System.UriParser.BuiltInUriParser}
            UserDrivenParsing   false   bool
            UserEscaped false   bool
            UserInfo    ""  string
    +       Static members      
            Server  "Apache/2.2.29 (Win32)" string
            StatusCode  OK  System.Net.HttpStatusCode
            StatusDescription   "OK"    string
    

    【讨论】:

    猜你喜欢
    • 2012-02-12
    • 2011-06-01
    • 2010-10-15
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多