使用 Selenium Htmlunit Java 的最终重定向 URL

Question

我有一个问题要问你，我认为可以使用 Selenium 解决这个问题 我有一组 URL，例如下面的 URL。

http://www.sears.com/search=little tikes&Little Tikes?filter=Brand&keywordSearch=false&vName=Toys+%26+Games&catalogId=12605&catPrediction=false&previousSort=ORIGINAL_SORT_ORDER&viewItems=50&storeId=10153&adCell=W3

如果您将 URL 原样粘贴在浏览器中，它将最终重定向到另一个 URL，您可以在浏览器的地址栏中进行验证（例如 Firefox）。 我需要获取重定向的 URL，无论重定向是否来自 javascript 代码 是否可以使用 selenium 框架来做到这一点？

我已经尝试过为此使用 HTMLUnit，但是我收到以下 javascript 执行错误。请帮忙！

com.gargoylesoftware.htmlunit.ScriptException: TypeError: Cannot call method "indexOf" of null (script in http://www.sears.com/search=little%20tikes&Little%20Tikes?filter=Brand&keywordSearch=false&catalogId=12605&adCell=W3&catPrediction=false&previousSort=ORIGINAL_SORT_ORDER&viewItems=50&storeId=10153&levels=Toys+%26+Games from (6942, 33) to (6974, 14)#6966)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:669) ~[htmlunit-2.12.jar:2.12]
    at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:601) ~[htmlunit-core-js-2.12.jar:?]
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:507) ~[htmlunit-core-js-2.12.jar:?]
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:555) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:530) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptIfPossible(HtmlPage.java:979) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.html.HtmlScript.executeInlineScriptIfNeeded(HtmlScript.java:337) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:415) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:266) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:276) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:676) ~[htmlunit-2.12.jar:2.12]
    at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source) ~[xercesImpl-2.10.0.jar:?]
    at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:635) ~[htmlunit-2.12.jar:2.12]
    at org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1170) ~[nekohtml-1.9.18.jar:1.9.18]
    at org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1072) ~[nekohtml-1.9.18.jar:1.9.18]
    at org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206) ~[nekohtml-1.9.18.jar:?]
    at org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:330) ~[nekohtml-1.9.18.jar:?]
    at org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3074) ~[nekohtml-1.9.18.jar:1.9.18]
    at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2041) ~[nekohtml-1.9.18.jar:1.9.18]
    at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:918) ~[nekohtml-1.9.18.jar:1.9.18]
    at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499) ~[nekohtml-1.9.18.jar:1.9.18]
    at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452) ~[nekohtml-1.9.18.jar:1.9.18]
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) ~[xercesImpl-2.10.0.jar:?]
    at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:892) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:241) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:187) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:268) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:156) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:434) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:309) ~[htmlunit-2.12.jar:2.12]
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:374) ~[htmlunit-2.12.jar:2.12]

Answer 1

如果我理解了你的问题，这应该很容易。以下是步骤

1. Get the FirefoxDriver object
2. call driver.get("http://www.sears.com/search=little tikes&Little Tikes?filter=Brand&keywordSearch=false&vName=Toys+%26+Games&catalogId=12605&catPrediction=false&previousSort=ORIGINAL_SORT_ORDER&viewItems=50&storeId=10153&adCell=W3");
This will open the url in firefox. On open the url will be forwarded to actual url. (This is as per my understanding from your description)
3. Then you can do driver.getCurrentUrl(). This will give you the url.

让我知道这是否适合你:)

更新：

        WebClient webClient = new WebClient(BrowserVersion.INTERNET_EXPLORER_9);
        webClient.getOptions().setJavaScriptEnabled(true);
        webClient.getOptions().setRedirectEnabled(true);
        webClient.getOptions().setThrowExceptionOnScriptError(false);
        webClient.getOptions().setCssEnabled(true);     
        HtmlPage page = (HtmlPage) webClient.getPage("http://www.sears.com/search=little tikes&Little Tikes?filter=Brand&keywordSearch=false&vName=Toys+%26+Games&catalogId=12605&catPrediction=false&previousSort=ORIGINAL_SORT_ORDER&viewItems=50&storeId=10153&adCell=W3");
        WebResponse response = page.getWebResponse();
        String content = response.getContentAsString();
        System.out.println(page.getUrl());

Answer 2

如果您使用的是 HTMLUnit Driver，请启用 JavaScript（默认设置为关闭），如下所示。

更多 HTMLUnit 使用 Rhino 作为它的 JavaScript 引擎，不同于其他主流浏览器 JS 引擎。

HtmlUnitDriver Browser_Session= new HtmlUnitDriver();
Browser_Session.setJavascriptEnabled(true);

or

HtmlUnitDriver Browser_Session = new HtmlUnitDriver(true);

以下步骤应获取重定向的 url。

Browser_Session.navigate().to("URL");
Browser_Session.getCurrentUrl(); //This fetches the current re-directed URL.

希望对你有帮助