目标服务器无法使用 htmlunit 和 tor 响应异常答案

【问题标题】：Target server failed to respond exception with htmlunit and tor目标服务器无法使用 htmlunit 和 tor 响应异常
【发布时间】：2012-07-02 08:11:05
【问题描述】：

我正在使用 htmlunit 向网站和 tor 发送请求以匿名。但是我得到了

目标服务器未能响应异常。我在google上搜索，发现如下代码。

HttpMethodRetryHandler myretryhandler = new HttpMethodRetryHandler() {
    public boolean retryMethod(
        final HttpMethod method, 
        final IOException exception, 
        int executionCount) {
        if (executionCount >= 5) {
            // Do not retry if over max retry count
            return false;
        }
        if (exception instanceof NoHttpResponseException) {
            // Retry if the server dropped connection on us
            return true;
        }
        if (!method.isRequestSent()) {
        // Retry if the request has not been sent fully or
        // if it's OK to retry methods that have been sent
        return true;
    }
    // otherwise do not retry
    return false;
    }
};

GetMethod httpget = new GetMethod("http://www.whatever.com/");
httpget.getParams().
setParameter(HttpMethodParams.RETRY_HANDLER, myretryhandler);
try {
    client.executeMethod(httpget);
    System.out.println(httpget.getStatusLine().toString());
} finally {
    httpget.releaseConnection();
}

但是我在 htmlunit 中找不到如何执行此操作。我怎样才能做到这一点？

【问题讨论】：

标签： java htmlunit tor

【解决方案1】：

提供的代码是一个 HttpClient 示例，如果您想使用 htmlunit，请访问他们的网站 http://htmlunit.sourceforge.net/ 并使用这些 sn-ps 您应该能够发送（发布？）请求

WebClient client = new WebClient(BrowserVersion.FIREFOX_3_6);
client.setTimeout(60000);
client.setRedirectEnabled(true);
client.setJavaScriptEnabled(true);
client.setThrowExceptionOnFailingStatusCode(false);
client.setThrowExceptionOnScriptError(false);
client.setCssEnabled(false);
client.setUseInsecureSSL(true);

    HtmlPage page = null;
    try {
        page = client.getPage("http://www.whatever.com");
    } catch (Exception e) {
        // TODO Auto-generated catch block
    }
    if (page.getWebResponse().getStatusCode() == 404) {
        System.out.println("Page not found");
    }

    // Post a request
    WebRequest request = new WebRequest(new URL("http://www.whatever.com/post_url"));
    request.setHttpMethod(HttpMethod.POST);
    List<NameValuePair> params = new ArrayList<NameValuePair>();
    params.add(new NameValuePair("login", userLogin));
    params.add(new NameValuePair("pass", userPassword));
    request.setRequestParameters(params);

    page = client.getPage(request);

【讨论】：

谢谢！我在 HTMLUnit 2.10 中遇到了这个确切的问题，上面的设置修复了“目标服务器未能响应异常”。我添加到我的代码中的唯一不存在的东西是 client.setRedirectEnabled(true);和 client.setUseInsecureSSL(true);

【解决方案2】：

    final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_45);
    webClient.getOptions().setTimeout(20000);
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
    webClient.getOptions().setCssEnabled(false);
    webClient.getOptions().setJavaScriptEnabled(true);
    webClient.getOptions().setUseInsecureSSL(true);
    webClient.getOptions().setRedirectEnabled(true);

    webClient.setConfirmHandler(new ConfirmHandler() {
        public boolean handleConfirm(Page page, String message) {
            return true;
        }
    });

    String binary = DatatypeConverter.printBase64Binary(auth.getBytes());
    webClient.addRequestHeader("Authorization", "Basic " + binary);

    //get General page
    final HtmlPage page = webClient.getPage("http://adress");

【讨论】：