HttpClient重定向到带有空格的URL引发异常

Question

我正在访问一个 URL，该 URL 将我重定向到其中包含空格的 URL。 （使用 HttpClient 4.x）如何防止这引发错误（用 %20 而不是 + 替换空格）

08-06 02:45:56.486: WARN/System.err(655): org.apache.http.client.ClientProtocolException
08-06 02:45:56.493: WARN/System.err(655):     at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:557)
08-06 02:45:56.534: WARN/System.err(655):     at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:509)
08-06 02:45:56.603: WARN/System.err(655):     at com.romcessed.romsearch.searchproviders.DopeRomsConnector$DownloadROMTask.doInBackground(DopeRomsConnector.java:636)
08-06 02:45:56.623: WARN/System.err(655):     at com.romcessed.romsearch.searchproviders.DopeRomsConnector$DownloadROMTask.doInBackground(DopeRomsConnector.java:1)
08-06 02:45:56.643: WARN/System.err(655):     at android.os.AsyncTask$2.call(AsyncTask.java:185)
08-06 02:45:56.663: WARN/System.err(655):     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
08-06 02:45:56.683: WARN/System.err(655):     at java.util.concurrent.FutureTask.run(FutureTask.java:137)
08-06 02:45:56.693: WARN/System.err(655):     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1068)
08-06 02:45:56.713: WARN/System.err(655):     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:561)
08-06 02:45:56.713: WARN/System.err(655):     at java.lang.Thread.run(Thread.java:1096)
08-06 02:45:56.743: WARN/System.err(655): Caused by: org.apache.http.ProtocolException: Invalid redirect URI: http://somewebsite.com/some file with spaces.zip
08-06 02:45:56.787: WARN/System.err(655):     at org.apache.http.impl.client.DefaultRedirectHandler.getLocationURI(DefaultRedirectHandler.java:116)
08-06 02:45:56.803: WARN/System.err(655):     at org.apache.http.impl.client.DefaultRequestDirector.handleResponse(DefaultRequestDirector.java:892)
08-06 02:45:56.813: WARN/System.err(655):     at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:457)
08-06 02:45:56.843: WARN/System.err(655):     at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:555)
08-06 02:45:56.843: WARN/System.err(655):     ... 9 more
08-06 02:45:56.873: WARN/System.err(655): Caused by: java.net.URISyntaxException: Illegal character in path at index #: http://somewebsite.com/some file with spaces.zip
08-06 02:45:56.913: WARN/System.err(655):     at java.net.URI$Helper.validatePath(URI.java:448)
08-06 02:45:56.923: WARN/System.err(655):     at java.net.URI$Helper.parseURI(URI.java:398)
08-06 02:45:56.953: WARN/System.err(655):     at java.net.URI$Helper.access$100(URI.java:302)
08-06 02:45:56.963: WARN/System.err(655):     at java.net.URI.<init>(URI.java:87)
08-06 02:45:56.993: WARN/System.err(655):     at org.apache.http.impl.client.DefaultRedirectHandler.getLocationURI(DefaultRedirectHandler.java:114)
08-06 02:45:57.013: WARN/System.err(655):     ... 12 more

Answer 1

Apache HTTP 库允许您注册一个 RedirectHandler 对象，该对象将在发生重定向时被调用。您可以使用它来拦截重定向并修复它。

（话虽如此，向您发送此重定向的网站已损坏。您应该联系他们并让他们知道。）

class CustomRedirectHandler extends DefaultRedirectHandler {
    public URI getLocationURI(HttpResponse response, HttpContext context) {
        // Extract the Location: header and manually convert spaces to %20's
        // Return the corrected URI
    }
}

DefaultHttpClient httpClient = new DefaultHttpClient();
RedirectHandler customRedirectHandler = new CustomRedirectHandler();
//...
httpClient.setRedirectHandler(customRedirectHandler);

Answer 2

这是我的工作代码:)

class spaceRedirectHandler extends DefaultRedirectHandler{

                private static final String REDIRECT_LOCATIONS = "http.protocol.redirect-locations";

                public spaceRedirectHandler() {
                    super();
                }

                public boolean isRedirectRequested(
                        final HttpResponse response,
                        final HttpContext context) {
                    if (response == null) {
                        throw new IllegalArgumentException("HTTP response may not be null");
                    }
                    int statusCode = response.getStatusLine().getStatusCode();
                    switch (statusCode) {
                    case HttpStatus.SC_MOVED_TEMPORARILY:
                    case HttpStatus.SC_MOVED_PERMANENTLY:
                    case HttpStatus.SC_SEE_OTHER:
                    case HttpStatus.SC_TEMPORARY_REDIRECT:
                        return true;
                    default:
                        return false;
                    } //end of switch
                }

                public URI getLocationURI(
                        final HttpResponse response, 
                        final HttpContext context) throws ProtocolException {
                    if (response == null) {
                        throw new IllegalArgumentException("HTTP response may not be null");
                    }
                    //get the location header to find out where to redirect to
                    Header locationHeader = response.getFirstHeader("location");
                    if (locationHeader == null) {
                        // got a redirect response, but no location header
                        throw new ProtocolException(
                                "Received redirect response " + response.getStatusLine()
                                + " but no location header");
                    }
//HERE IS THE MODIFIED LINE OF CODE
                    String location = locationHeader.getValue().replaceAll (" ", "%20");

                    URI uri;
                    try {
                        uri = new URI(location);            
                    } catch (URISyntaxException ex) {
                        throw new ProtocolException("Invalid redirect URI: " + location, ex);
                    }

                    HttpParams params = response.getParams();
                    // rfc2616 demands the location value be a complete URI
                    // Location       = "Location" ":" absoluteURI
                    if (!uri.isAbsolute()) {
                        if (params.isParameterTrue(ClientPNames.REJECT_RELATIVE_REDIRECT)) {
                            throw new ProtocolException("Relative redirect location '" 
                                    + uri + "' not allowed");
                        }
                        // Adjust location URI
                        HttpHost target = (HttpHost) context.getAttribute(
                                ExecutionContext.HTTP_TARGET_HOST);
                        if (target == null) {
                            throw new IllegalStateException("Target host not available " +
                                    "in the HTTP context");
                        }

                        HttpRequest request = (HttpRequest) context.getAttribute(
                                ExecutionContext.HTTP_REQUEST);

                        try {
                            URI requestURI = new URI(request.getRequestLine().getUri());
                            URI absoluteRequestURI = URIUtils.rewriteURI(requestURI, target, true);
                            uri = URIUtils.resolve(absoluteRequestURI, uri); 
                        } catch (URISyntaxException ex) {
                            throw new ProtocolException(ex.getMessage(), ex);
                        }
                    }

                    if (params.isParameterFalse(ClientPNames.ALLOW_CIRCULAR_REDIRECTS)) {

                        RedirectLocations redirectLocations = (RedirectLocations) context.getAttribute(
                                REDIRECT_LOCATIONS);

                        if (redirectLocations == null) {
                            redirectLocations = new RedirectLocations();
                            context.setAttribute(REDIRECT_LOCATIONS, redirectLocations);
                        }

                        URI redirectURI;
                        if (uri.getFragment() != null) {
                            try {
                                HttpHost target = new HttpHost(
                                        uri.getHost(), 
                                        uri.getPort(),
                                        uri.getScheme());
                                redirectURI = URIUtils.rewriteURI(uri, target, true);
                            } catch (URISyntaxException ex) {
                                throw new ProtocolException(ex.getMessage(), ex);
                            }
                        } else {
                            redirectURI = uri;
                        }

                        if (redirectLocations.contains(redirectURI)) {
                            throw new CircularRedirectException("Circular redirect to '" +
                                    redirectURI + "'");
                        } else {
                            redirectLocations.add(redirectURI);
                        }
                    }

                    return uri;
                }
        }

Answer 3

我建议创建自定义重定向策略

class CustomRedirectStrategy extends DefaultRedirectStrategy {
  // NOTE: Hack for bad redirects such as: http://www.healio.com/Rss/Allergy%20Immunology
  override def createLocationURI(location: String): URI = {
    try {
      super.createLocationURI(location)
    } catch {
      case ex: ProtocolException =>
        val url = new URL(location)
        val uri = new URI(url.getProtocol, url.getUserInfo, url.getHost, url.getPort, url.getPath, url.getQuery, url.getRef)
        uri
    }
  }
}

您可以通过setRedirectStrategy 方法在您的客户端中设置。

HttpAsyncClients.custom.setRedirectStrategy(new CustomRedirectStrategy).build

Answer 4

另一个使用 %20 替换空格的工作代码示例，基于 https://stackoverflow.com/a/8962879/956415

private download(){

    ...

    mHttpClient = new DefaultHttpClient(httpParams);

    mHttpClient.setRedirectHandler(new DefaultRedirectHandler() {
        @Override
        public boolean isRedirectRequested(HttpResponse httpResponse, HttpContext httpContext) {
            return super.isRedirectRequested(httpResponse, httpContext);
        }

        @Override
        public URI getLocationURI(HttpResponse httpResponse, HttpContext httpContext) throws ProtocolException {
            return sanitizeUrl(httpResponse.getFirstHeader("location").getValue());
        }
    });


}

private URI sanitizeUrl(String sanitizeURL) throws ProtocolException {

    URI uri = null;

    try {
        URL url = new URL(URLDecoder.decode(sanitizeURL, UTF_8));
        // https://stackoverflow.com/a/8962879/956415
        uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(), url.getPort(), url.getPath(), url.getQuery(), url.getRef());
    } catch (URISyntaxException | MalformedURLException | UnsupportedEncodingException e) {
        throw new ProtocolException(e.getMessage(), e);
    }

    return uri;
}

Answer 5

如果有人在 2020 年及以后寻找完整的解决方案，请参见下文。扩展 DefaultRedirectStrategy 并覆盖创建重定向 URI 的方法以将空格转换为 %20 序列。

public class CleanUrlRedirectStrategy extends DefaultRedirectStrategy
{
    private static final Logger logger = LoggerFactory.getLogger( MethodHandles.lookup().lookupClass() );

    // Clean spaces in redirect URLs to prevent an IOException
    @Override
    protected URI createLocationURI( final @NotNull String orig ) throws ProtocolException
    {
        // Replace spaces
        Objects.requireNonNull( orig );
        var fixed = orig.replaceAll("\\s","%20"); // Not \\s+
        if ( ! orig.equals( fixed ) )
        {
            logger.debug( "Cleaned redirect URL from '{}' to '{}'", orig, fixed );
        }

        return super.createLocationURI( fixed );
    }
}

和

var httpClient = HttpClientBuilder
    .create()
    .setRedirectStrategy( new CleanUrlRedirectStrategy() )
    .build();

Answer 6

http://download.oracle.com/javase/6/docs/api/java/net/URI.html

或者只是将“+”替换为 %20

类似问题

In a URL, should spaces be encoded using %20 or +?