改造 + okhttp ：检索 GZIPInputStream答案

【问题标题】：retrofit + okhttp : Retrieve GZIPInputStream改造 + okhttp ：检索 GZIPInputStream
【发布时间】：2014-03-02 08:41:18
【问题描述】：

当我使用改造 1.4.1 和 okhttp 1.3.0 在 WS 上激活 gzip 时遇到问题。

RequestInterceptor requestInterceptor = new RequestInterceptor() {
            @Override
            public void intercept(RequestFacade request) {
                request.addHeader("content-type", "application/json");
                request.addHeader("accept-encoding", "gzip");  // Here is the problem
            }
        }; 
RestAdapter restAdapter = new RestAdapter.Builder()
            .setEndpoint(Constants.HOST)
            .setLogLevel(RestAdapter.LogLevel.FULL)
            .setRequestInterceptor(requestInterceptor)
            .build();

如果我评论以下行 request.addHeader("accept-encoding", "gzip"); 没有问题，但如果 gzip 被激活，我得到一个错误（我的请求属于 failure）。

这是我的日志猫request.addHeader("accept-encoding", "gzip");

1326               Retrofit  D  : HTTP/1.1 200 OK
  1326               Retrofit  D  Cache-Control: public, max-age=600
  1326               Retrofit  D  Content-Encoding: gzip
  1326               Retrofit  D  Content-Length: 254
  1326               Retrofit  D  Content-Type: application/json
  1326               Retrofit  D  Date: Wed, 05 Feb 2014 20:22:26 GMT
  1326               Retrofit  D  OkHttp-Received-Millis: 1391631746193
  1326               Retrofit  D  OkHttp-Response-Source: NETWORK 200
  1326               Retrofit  D  OkHttp-Selected-Transport: http/1.1
  1326               Retrofit  D  OkHttp-Sent-Millis: 1391631745971
  1326               Retrofit  D  Server: Apache
  1326               Retrofit  D  Vary: Accept-Encoding
  1326               Retrofit  D  X-Powered-By: PHP/5.3.3-7+squeeze18
  1326               Retrofit  D  ������������}�?O�0��~����nHZOH0 �D�ù���?���~w.�:����=�{�
                               ����|A���=�V/~}o�)���&����<�`�6&��ѳ:��5�ke��V�WD�H�
                               ���ud�J5رyp��G�ːg�y�ʴ����Mxq<�#�Rb`Su�@�0��y��lr;�W�2�C3�
                               T��$���.�
                                          ��xѥ���R
                                                   y���hmt����R����o����v��7@P�
                               4Y����
  1326               Retrofit  D  <--- END HTTP (254-byte body)
  1326             System.err  W  retrofit.RetrofitError: retrofit.converter.ConversionException: com.google.gson.JsonSyntaxException: java.lang.Ille
                               galStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1
  1326             System.err  W  at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:408)
  1326             System.err  W  at retrofit.RestAdapter$RestHandler.access$100(RestAdapter.java:262)
  1326             System.err  W  at retrofit.RestAdapter$RestHandler$2.obtainResponse(RestAdapter.java:313)
  1326             System.err  W  at retrofit.CallbackRunnable.run(CallbackRunnable.java:38)
  1326             System.err  W  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
  1326             System.err  W  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
  1326             System.err  W  at retrofit.Platform$Android$2$1.run(Platform.java:136)
  1326             System.err  W  at java.lang.Thread.run(Thread.java:841)
  1326             System.err  W  Caused by: retrofit.converter.ConversionException: com.google.gson.JsonSyntaxException: java.lang.IllegalStateExcep

如何开启 gzip？

提前谢谢

【问题讨论】：

标签： android retrofit okhttp

【解决方案1】：

只需从您的代码中省略 accept-encoding 标头即可。 OkHttp 会添加自己的accept-encoding 标头，如果服务器使用 gzip 响应，那么 OkHttp 会默默地为您解压缩。

【讨论】：

【解决方案2】：

遇到类似问题后（在我的情况下，没有添加任何 Accept-Encoding 标头，它偶尔会无法解压缩响应，还会在其中留下 Content-Encoding: gzip 标头，导致 JSON 解析器崩溃），并且没有明确的解决方法，我通过创建下面的委托客户端实现手动启用了改造的 gzip。它工作得很好，除了你可能不应该将它用于非常大（例如 > 250KB）的响应，因为它们首先被复制到一个字节数组中。

public class GzippedClient implements Client {

    private Client wrappedClient;

    public GzippedClient(Client wrappedClient) {
        this.wrappedClient = wrappedClient;
    }

    @Override
    public Response execute(Request request) throws IOException {
        Response response = wrappedClient.execute(request);

        boolean gzipped = false;
        for (Header h : response.getHeaders()) {
            if (h.getName() != null && h.getName().toLowerCase().equals("content-encoding") && h.getValue() != null && h.getValue().toLowerCase().equals("gzip")) {
                gzipped = true;
                break;
            }
        }

        Response r = null;
        if (gzipped) {
            InputStream is = null;
            ByteArrayOutputStream bos = null;

            try {
                is = new BufferedInputStream(new GZIPInputStream(response.getBody().in()));
                bos = new ByteArrayOutputStream();

                int b;
                while ((b = is.read()) != -1) {
                    bos.write(b);
                }

                TypedByteArray body = new TypedByteArray(response.getBody().mimeType(), bos.toByteArray());
                r = new Response(response.getUrl(), response.getStatus(), response.getReason(), response.getHeaders(), body);
            } finally {
                if (is != null) {
                    is.close();
                }
                if (bos != null) {
                    bos.close();
                }
            }
        } else {
            r = response;
        }
        return r;
    }

}

您还必须在请求中添加 Accept-Encoding 标头，例如通过使用 RequestInterceptor

requestFacade.addHeader("Accept-Encoding", "gzip");

最后，您必须将现有的 Client 包装到这个新的 GzippedClient 中，如下所示：

restBuilder.setClient(new GzippedClient(new OkClient(okHttpClient)));

就是这样。现在您的数据将被压缩。

编辑：似乎在 OkHttp 版本 1.5.1 中，似乎已修复与透明 gzipping 相关的错误 (https://github.com/square/okhttp/pull/632)，这可能（或可能不是）是我最初问题的根源。如果是这样，可能不再发生 un-gzip 偶尔失败的情况，尽管这种情况发生得很少，我还不能确认这一点。无论哪种方式，如果您想依靠自己，而不是透明地添加/删除标题和 gzip，那么所描述的解决方案将起作用。

【讨论】：

【解决方案3】：

如果您在 OkHttp 库中检查 HttpEngine，那么您可以找到以下代码，这意味着如果您在请求中手动添加“Accept-Encoding”：“gzip”标头，则解压缩是您的责任。 /** * True if this client added an "Accept-Encoding: gzip" header field and is * therefore responsible for also decompressing the transfer stream. */ private boolean transparentGzip;

因此，如果您手动添加“Accept-Encoding”：“gzip”标头，则在收到响应后，请按照以下方式解压缩。

private static String readContentFromTypedInput(TypedInput typedInput){

        InputStreamReader isr = null;
        BufferedReader br = null;
        char[] cbuf = new char[512];
        StringWriter stringWriter = new StringWriter();

        try {
            final InputStream in = typedInput.in();
            boolean isGzipped = GZipper.isGzippped(in);
            if(isGzipped){
                return new String(GZipper.doUnZip(in));
            }
            isr = new InputStreamReader(in);
            br = new BufferedReader(isr);
            while((br.read(cbuf))!= -1){
                stringWriter.write(cbuf);
            }
        } catch (IOException e) {
            throw new InvalidTestCaseException("failed read received content.", e);
        } finally{
            try{
                if(br != null) br.close();
            }catch(IOException e){ 
                //ignore
            }
        }
        return stringWriter.toString().trim();
    }

GZipper.java

public class GZipper{

public static final String DEFAULT_CHARSET = "utf-8";
private static final int BYTE_BLOCK_LENGTH = 1024;

public static byte[] doZip(final String message){
    if(message == null || message.isEmpty()){
        throw new SystemFailedException("Fail to zip - given message is null or empty");
    }
    byte[] gzippped = null;
    try {
        gzippped = doZip(message.getBytes(DEFAULT_CHARSET));
    } catch (Throwable e) {
        throw new SystemFailedException(e.getMessage(), e);
    }
    return gzippped;
}

public static byte[] doZip(final byte[] unzippedMessageByte){
    validate(unzippedMessageByte, "Fail to zip - given bytes is null or empty");

    ByteArrayInputStream is = null;
    ByteArrayOutputStream bos = null;
    GZIPOutputStream gzip_os = null;
    byte[] compressedBytes = null;
    try{
        is = new ByteArrayInputStream(unzippedMessageByte);
        bos = new ByteArrayOutputStream();
        gzip_os = new GZIPOutputStream(bos);
        copy(is, gzip_os);
        gzip_os.finish();
        compressedBytes = bos.toByteArray();
    }catch(IOException e){
        throw new SystemFailedException(e.getMessage(), e);
    }finally{
        try{
            if(is != null){is.close();}
            if(gzip_os != null){gzip_os.close();}
            if(bos != null){bos.close();}
        }catch(IOException e){
            //ignore
        }
    }
    return compressedBytes;
}

public static String doUnZipToString(final byte[] gzippedMessage){
    validate(gzippedMessage, "Fail to unzip - given bytes is null or empty");
    byte[] gzippped = null;
    String unzippedMessage = null;
    try {
        gzippped = doUnZip(gzippedMessage);
        unzippedMessage = new String(gzippped, DEFAULT_CHARSET);
    } catch (Throwable e) {
        throw new SystemFailedException(e.getMessage(), e);
    }
    return unzippedMessage;
}

private static void validate(final byte[] bytes, String failedMessage) {
    if(bytes == null || bytes.length == 0){
        throw new SystemFailedException(failedMessage);
    }
}

public static byte[] doUnZip(InputStream in) {
    if(!(in instanceof ByteArrayInputStream)){
        try {
            return doUnZip(IOUtils.toByteArray(in));
        } catch (IOException e) {
            throw new SystemFailedException(e.getMessage(), e);
        }
    }

     ByteArrayOutputStream bos = null;
     InputStream gzip_is = null;
     byte[] bytes = null;
    try{
        bos = new ByteArrayOutputStream();
        gzip_is = new GZIPInputStream(in);
        copy(gzip_is,bos);
        bytes = bos.toByteArray();
    }catch(IOException e){
        throw new SystemFailedException(e.getMessage(), e);
    }finally{
        try{
            if(gzip_is != null) gzip_is.close();
            if(bos != null) bos.close();
        }catch(IOException e){
            //ignore
        }
    }
    return bytes;
}

public static byte[] doUnZip(final byte[] zippedMessage){
    validate(zippedMessage, "Fail to unzip - given bytes is null or empty");
    ByteArrayInputStream is = null;
    try{
        is = new ByteArrayInputStream(zippedMessage);
        return doUnZip(is);
    }finally{
        try{
            if(is != null) is.close();
        }catch(IOException e){
            //ignore
        }
    }
}

public static String doUnZip(File file){
    validate(file);

    GZIPInputStream gzipInputStream = null;
    StringWriter writer = null;
    String result = "";

    try{
        byte[] buffer = new byte[BYTE_BLOCK_LENGTH];
        gzipInputStream = new GZIPInputStream(new FileInputStream(file));
        writer = new StringWriter();
        while((gzipInputStream.read(buffer)) > 0){
            writer.write(new String(buffer));
            writer.flush();
        }
        result = writer.toString();
    }catch(IOException e){
        //do something to handle exception
    }
    finally{
        try{
            if(writer != null){writer.close();}
            if(gzipInputStream != null){gzipInputStream.close();}
        }catch(IOException e){
            //ignore
        }
    }
    return result;
}

private static void validate(File file) {
    if(file==null || !file.exists()){
        throw new SystemFailedException("Fail to unzip - file is not exist");
    }
}

private static void copy(InputStream in, OutputStream out)throws IOException {
    byte[] buf = new byte[BYTE_BLOCK_LENGTH];
    int len = -1;
    while ((len = in.read(buf, 0, buf.length)) != -1) {
        out.write(buf, 0, len);
    }
}

public static boolean isGzipped(byte[] input){
    return isGzippped(new ByteArrayInputStream(input));
}

public static boolean isGzippped(InputStream in){
    boolean markSupported = in.markSupported();
    boolean result = false;
    try {
        if(markSupported){
            in.mark(0);
            result = (readUShort(in) == GZIPInputStream.GZIP_MAGIC);
            in.reset();
        }
    } catch (Exception e) {
        result = false;
    }
    return result;
}

private static int readUShort(InputStream in) throws IOException {
    int b = readUByte(in);
    return ((int)readUByte(in) << 8) | b;
}

/*
 * Reads unsigned byte.
 */
private static int readUByte(InputStream in) throws IOException {
    int b = in.read();
    if (b == -1) {
        throw new EOFException();
    }
    if (b < -1 || b > 255) {
       b = 0;
    }
    return b;
}

}

【讨论】：

【解决方案4】：

你需要使用同样由 Square 制作的 Okhttp，它支持抓握。不知道您是否创建了自定义实例，或者它是否默认启用，您应该检查文档。

【讨论】：