Geckodriver/Selenium 3.x 似乎不喜欢代理答案

【问题标题】：Geckodriver/Selenium 3.x doesn't seem to like proxiesGeckodriver/Selenium 3.x 似乎不喜欢代理
【发布时间】：2018-03-15 02:15:23
【问题描述】：

好的，我又回来了……带着另一个 Selenium 3.x/geckodriver 问题。

我目前有一个设置，我通过 browsermob-proxy 路由来自我的 selenium 测试的所有网络流量，所以我的测试要求我给他们代理信息路由所有这些数据，以便稍后我可以查询它。

我目前遇到的问题是，无论我如何尝试设置代理，我的测试都会出现不同的异常。

尝试1：通过socks设置代理

profile.setPreference("network.proxy.type", 1);
profile.setPreference("network.proxy.socks", "localhost");
profile.setPreference("network.proxy.socks_port", 9091);

此尝试导致以下堆栈跟踪：

    Caused by: org.openqa.selenium.SessionNotCreatedException:
InvalidArgumentError: Expected [object Undefined] undefined to be an integer
Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:42:44.417Z'
System info: host: 'Administrators-MacBook-Pro.local', ip: 'fe80:0:0:0:a299:9bff:fe05:a43b%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.6', java.version: '1.8.0_77'
Driver info: driver.version: unknown
remote stacktrace: stack backtrace:
   0:        0x10d573754 - backtrace::backtrace::trace::h8e94df436d083743
   1:        0x10d573b2f - backtrace::capture::Backtrace::new::he142908a5a8f9eda
   2:        0x10d203cbf - webdriver::error::WebDriverError::new::h160d25bec9197da5
   3:        0x10d211442 - geckodriver::marionette::MarionetteSession::response::h970369545ccb2ae5
   4:        0x10d2205aa - geckodriver::marionette::MarionetteConnection::send_command::h4b272021662c39d8
   5:        0x10d20fd48 - _$LT$geckodriver..marionette..MarionetteHandler$u20$as$u20$webdriver..server..WebDriverHandler$LT$geckodriver..marionette..GeckoExtensionRoute$GT$$GT$::handle_command::h2c7905f1d7536e5d
   6:        0x10d198d43 - _$LT$webdriver..server..Dispatcher$LT$T$C$$u20$U$GT$$GT$::run::hd1e66b7b65330ddf
   7:        0x10d204b1a - webdriver::server::start::_$u7b$$u7b$closure$u7d$$u7d$::h690494072298ecdc
   8:        0x10d0fcbaa - std::sys_common::backtrace::__rust_begin_short_backtrace::h9a6ad1a465c9c77e
   9:        0x10d10dd83 - std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hb084b9313dc657eb
  10:        0x10d0bbb7a - _$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h9e0d4487cab44eb7
  11:        0x10d10e8b9 - std::panicking::try::do_call::hd4a4ecf8d08f7d4d
  12:        0x10d89567c - __rust_maybe_catch_panic
  13:        0x10d10e6ec - std::panicking::try::hdaf1b8a3b363a07e
  14:        0x10d10b5e5 - std::panic::catch_unwind::h9643785660d717d4
  15:        0x10d10da3e - std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::hc86bbe5d0f3f926a
  16:        0x10d17b663 - _$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::h9cb8abeb542e88ca
  17:        0x10d8918eb - std::sys::imp::thread::Thread::new::thread_start::h2d2a40d6b00ba029
  18:     0x7fff99df499c - _pthread_body
  19:     0x7fff99df4919 - _pthread_start
Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:42:44.417Z'
System info: host: 'Administrators-MacBook-Pro.local', ip: 'fe80:0:0:0:a299:9bff:fe05:a43b%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.6', java.version: '1.8.0_77'
Driver info: driver.version: unknown

尝试 2：通过 JSON 设置

JsonObject json = new JsonObject();
json.addProperty("proxyType", "MANUAL");
json.addProperty("httpProxy", "127.0.0.1");
json.addProperty("httpProxyPort", 9091);
json.addProperty("sslProxy", "127.0.0.1");
json.addProperty("sslProxyPort", 9091);

capabilities.setCapability("proxy", json);

这会导致以下堆栈跟踪：

Caused by: org.openqa.selenium.InvalidArgumentException:
httpProxyPort was not a valid proxy configuration capability
Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:42:44.417Z'
System info: host: 'Administrators-MacBook-Pro.local', ip: 'fe80:0:0:0:a299:9bff:fe05:a43b%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.6', java.version: '1.8.0_77'
Driver info: driver.version: unknown
remote stacktrace: stack backtrace:
   0:        0x107004754 - backtrace::backtrace::trace::h8e94df436d083743
   1:        0x107004b2f - backtrace::capture::Backtrace::new::he142908a5a8f9eda
   2:        0x106d19c3f - webdriver::error::WebDriverError::new::h3c5206efc2aea993
   3:        0x106d07fe9 - webdriver::capabilities::SpecNewSessionParameters::validate_proxy::h0d2c552864af830d
   4:        0x106c93b2b - webdriver::capabilities::SpecNewSessionParameters::validate::ha71de94fac9331d9
   5:        0x106b509e6 - _$LT$webdriver..capabilities..SpecNewSessionParameters$u20$as$u20$webdriver..capabilities..CapabilitiesMatching$GT$::match_browser::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h1c32cd867ef807c3
   6:        0x106bbf37b - _$LT$core..result..Result$LT$T$C$$u20$E$GT$$GT$::and_then::hf76bfa0e307d12b7
   7:        0x106b4fd44 - _$LT$webdriver..capabilities..SpecNewSessionParameters$u20$as$u20$webdriver..capabilities..CapabilitiesMatching$GT$::match_browser::_$u7b$$u7b$closure$u7d$$u7d$::ha440f562183a4100
   8:        0x106beb870 - core::ops::function::impls::_$LT$impl$u20$core..ops..function..FnOnce$LT$A$GT$$u20$for$u20$$RF$$u27$a$u20$mut$u20$F$GT$::call_once::h97d995e0820d8c8b
   9:        0x106b80fd5 - _$LT$core..option..Option$LT$T$GT$$GT$::map::hec1c45e28df3f669
  10:        0x106c7ed4c - _$LT$core..iter..Map$LT$I$C$$u20$F$GT$$u20$as$u20$core..iter..iterator..Iterator$GT$::next::h3e80bbc98e18278b
  11:        0x106b6b156 - _$LT$$LT$core..result..Result$LT$V$C$$u20$E$GT$$u20$as$u20$core..iter..traits..FromIterator$LT$core..result..Result$LT$A$C$$u20$E$GT$$GT$$GT$..from_iter..Adapter$LT$Iter$C$$u20$E$GT$$u20$as$u20$core..iter..iterator..Iterator$GT$::next::hef0037ea92f136e0
  12:        0x106c69db6 - _$LT$$RF$$u27$a$u20$mut$u20$I$u20$as$u20$core..iter..iterator..Iterator$GT$::next::h13a60fd6da24f240
  13:        0x106c84069 - _$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$alloc..vec..SpecExtend$LT$T$C$$u20$I$GT$$GT$::from_iter::h3c5cd3bebd0b7a94
  14:        0x106c85ebb - _$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..iter..traits..FromIterator$LT$T$GT$$GT$::from_iter::h0a9ec0a7d8599d2f
  15:        0x106b5517b - _$LT$core..result..Result$LT$V$C$$u20$E$GT$$u20$as$u20$core..iter..traits..FromIterator$LT$core..result..Result$LT$A$C$$u20$E$GT$$GT$$GT$::from_iter::h3912ad22da5fa8a1
  16:        0x106c05d06 - core::iter::iterator::Iterator::collect::h37a3ccf2360609f5
  17:        0x106b4f700 - _$LT$webdriver..capabilities..SpecNewSessionParameters$u20$as$u20$webdriver..capabilities..CapabilitiesMatching$GT$::match_browser::h68cf564270268ccf
  18:        0x106b4e5f5 - _$LT$webdriver..command..NewSessionParameters$u20$as$u20$webdriver..capabilities..CapabilitiesMatching$GT$::match_browser::hbc7b5070d6ac176e
  19:        0x106c9dcb7 - geckodriver::marionette::MarionetteHandler::create_connection::h48a7098743d092a0
  20:        0x106ca0a71 - _$LT$geckodriver..marionette..MarionetteHandler$u20$as$u20$webdriver..server..WebDriverHandler$LT$geckodriver..marionette..GeckoExtensionRoute$GT$$GT$::handle_command::h2c7905f1d7536e5d
  21:        0x106c29d43 - _$LT$webdriver..server..Dispatcher$LT$T$C$$u20$U$GT$$GT$::run::hd1e66b7b65330ddf
  22:        0x106c95b1a - webdriver::server::start::_$u7b$$u7b$closure$u7d$$u7d$::h690494072298ecdc
  23:        0x106b8dbaa - std::sys_common::backtrace::__rust_begin_short_backtrace::h9a6ad1a465c9c77e
  24:        0x106b9ed83 - std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hb084b9313dc657eb
  25:        0x106b4cb7a - _$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h9e0d4487cab44eb7
  26:        0x106b9f8b9 - std::panicking::try::do_call::hd4a4ecf8d08f7d4d
  27:        0x10732667c - __rust_maybe_catch_panic
  28:        0x106b9f6ec - std::panicking::try::hdaf1b8a3b363a07e
  29:        0x106b9c5e5 - std::panic::catch_unwind::h9643785660d717d4
  30:        0x106b9ea3e - std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::hc86bbe5d0f3f926a
  31:        0x106c0c663 - _$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::h9cb8abeb542e88ca
  32:        0x1073228eb - std::sys::imp::thread::Thread::new::thread_start::h2d2a40d6b00ba029
  33:     0x7fff99df499c - _pthread_body
  34:     0x7fff99df4919 - _pthread_start
Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:42:44.417Z'
System info: host: 'Administrators-MacBook-Pro.local', ip: 'fe80:0:0:0:a299:9bff:fe05:a43b%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.6', java.version: '1.8.0_77'
Driver info: driver.version: unknown

尝试3：通过setPreference设置，不带socks

profile.setPreference("network.proxy.type", 1);
profile.setPreference("network.proxy.http", "localhost");
profile.setPreference("network.proxy.http_port", 9091);

这会导致以下堆栈跟踪：

Caused by: org.openqa.selenium.SessionNotCreatedException:
InvalidArgumentError: Expected [object Undefined] undefined to be an integer
Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:42:44.417Z'
System info: host: 'Administrators-MacBook-Pro.local', ip: 'fe80:0:0:0:a299:9bff:fe05:a43b%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.6', java.version: '1.8.0_77'
Driver info: driver.version: unknown
remote stacktrace: stack backtrace:
   0:        0x10f5b6754 - backtrace::backtrace::trace::h8e94df436d083743
   1:        0x10f5b6b2f - backtrace::capture::Backtrace::new::he142908a5a8f9eda
   2:        0x10f246cbf - webdriver::error::WebDriverError::new::h160d25bec9197da5
   3:        0x10f254442 - geckodriver::marionette::MarionetteSession::response::h970369545ccb2ae5
   4:        0x10f2635aa - geckodriver::marionette::MarionetteConnection::send_command::h4b272021662c39d8
   5:        0x10f252d48 - _$LT$geckodriver..marionette..MarionetteHandler$u20$as$u20$webdriver..server..WebDriverHandler$LT$geckodriver..marionette..GeckoExtensionRoute$GT$$GT$::handle_command::h2c7905f1d7536e5d
   6:        0x10f1dbd43 - _$LT$webdriver..server..Dispatcher$LT$T$C$$u20$U$GT$$GT$::run::hd1e66b7b65330ddf
   7:        0x10f247b1a - webdriver::server::start::_$u7b$$u7b$closure$u7d$$u7d$::h690494072298ecdc
   8:        0x10f13fbaa - std::sys_common::backtrace::__rust_begin_short_backtrace::h9a6ad1a465c9c77e
   9:        0x10f150d83 - std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hb084b9313dc657eb
  10:        0x10f0feb7a - _$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h9e0d4487cab44eb7
  11:        0x10f1518b9 - std::panicking::try::do_call::hd4a4ecf8d08f7d4d
  12:        0x10f8d867c - __rust_maybe_catch_panic
  13:        0x10f1516ec - std::panicking::try::hdaf1b8a3b363a07e
  14:        0x10f14e5e5 - std::panic::catch_unwind::h9643785660d717d4
  15:        0x10f150a3e - std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::hc86bbe5d0f3f926a
  16:        0x10f1be663 - _$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::h9cb8abeb542e88ca
  17:        0x10f8d48eb - std::sys::imp::thread::Thread::new::thread_start::h2d2a40d6b00ba029
  18:     0x7fff99df499c - _pthread_body
  19:     0x7fff99df4919 - _pthread_start
Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:42:44.417Z'
System info: host: 'Administrators-MacBook-Pro.local', ip: 'fe80:0:0:0:a299:9bff:fe05:a43b%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.6', java.version: '1.8.0_77'
Driver info: driver.version: unknown

使用 geckodriver 和 selenium 设置代理的适当方法是什么？

我正在使用 Selenium 3.5.3 和 geckodriver 0.19

提前致谢！

编辑：

这是我用来对 firefox 和其他浏览器进行这些修改的整个方法.....我正在使用 Selion，所以这也可能是一个因素。

package com.iacapps.ste.ta.helpers;

import com.google.common.base.Strings;
import com.paypal.selion.platform.grid.Grid;
import com.paypal.selion.platform.grid.browsercapabilities.DefaultCapabilitiesBuilder;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxProfile;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Collections;

public class CustomCapabilities extends DefaultCapabilitiesBuilder
{
  private static final Logger logger = LoggerFactory.getLogger(DefaultCapabilitiesBuilder.class);

  private static final String SAUCE_TUNNEL_PROPERTY = "sauceTunnel";
  private static final String SAUCE_ENABLED_PROPERTY = "enableSauceConnect";
  private static final String TUNNEL_CAPABILITY = "tunnelIdentifier";
  private static final String ACCEPT_ALL_SSL_CAPABILITY = "acceptSslCerts";
  private static final String CHROME_SWITCHES = "chrome.switches";
  private static final String CHROME_IGNORE_SSL = "--ignore-certificate-errors";
  private static final String FIREFOX_ACCEPT_BAD_CERTS_CAPABILITY = "acceptInsecureCerts";

  @Override
  public DesiredCapabilities getCapabilities(DesiredCapabilities capabilities)
  {
    String sauceEnabledValue = System.getProperty(SAUCE_ENABLED_PROPERTY);
    String tunnelIdValue = System.getProperty(SAUCE_TUNNEL_PROPERTY);
    //This will just prevent the warning being printed when sauceconnect isn't enabled.
    if (!Strings.isNullOrEmpty(sauceEnabledValue) && Boolean.valueOf(sauceEnabledValue))
    {
      if (Strings.isNullOrEmpty(tunnelIdValue))
      {
        logger.warn("{} not set", SAUCE_TUNNEL_PROPERTY);
      }
      else
      {
        // TODO:  Maybe move this to non-firefox browsers??
        //capabilities.setCapability(TUNNEL_CAPABILITY, tunnelIdValue);
      }
    }


    // Seems like with the latest version of selenium (3.x) we need to be kinda specific about
    // making sure that we're going to pass the right configuration crap to the particular drivers.
    // So for now, I'm just going to make some broad strokes and make things so that we know
    // if we're on firefox or "not firefox".  This might need to be changed/updated as we see that
    // other browsers aren't going to play nice with this, but well.  I hope it fixes it for now :)

    if (!Grid.getWebTestSession().getBrowser().contains("firefox"))
    {

      //There's a reason for this charlie foxtrot.  I don't always get to know what browser driver I'm
      //talking to.
      //Per selenium docs: "Whether the session should accept all SSL certs by default."
      //The DOWNSIDE: this seems to work with newer browser drivers, but it may not work with old ones.
      capabilities.setCapability(ACCEPT_ALL_SSL_CAPABILITY, true);
      //This *supposedly* works with some versions of IE.
      capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
      //This *supposedly* works with some chrome versions.

      capabilities.setCapability(CHROME_SWITCHES, Collections.singletonList(CHROME_IGNORE_SSL));
    }


    if (Grid.getWebTestSession().getBrowser().contains("firefox"))
    {

          //Oh my god please work you STUPID FIREFOX
          //http://stackoverflow.com/a/40788935
          //https://bugzilla.mozilla.org/show_bug.cgi?id=1103196
          //Should work with firefox > v51
          capabilities.setCapability(FIREFOX_ACCEPT_BAD_CERTS_CAPABILITY,true);
          //When in doubt SET EVERYTHING!


          FirefoxProfile profile = new FirefoxProfile();

          // Okay, let's try this socks stuff
          // Okay, socks doesn't work either.  Same object undefined nonsense.
          /*profile.setPreference("network.proxy.type", 1);
          profile.setPreference("network.proxy.socks", "localhost");
          profile.setPreference("network.proxy.socks_port", 9091);*/


          // When attempting to set via profile, I get this
          // InvalidArgumentError: Expected [object Undefined] undefined to be an integer
          profile.setPreference("network.proxy.type", 1);
          profile.setPreference("network.proxy.http", "127.0.0.1");
          profile.setPreference("network.proxy.http_port", 9091);


          // JSON Type 1.
          // So with the JSON object I get a httpProxyPort not a valid option....
          /*JsonObject json = new JsonObject();
          json.addProperty("proxyType", "MANUAL");
          json.addProperty("httpProxy", "127.0.0.1");
          json.addProperty("httpProxyPort", 9091);
          json.addProperty("sslProxy", "127.0.0.1");
          json.addProperty("sslProxyPort", 9091);

          capabilities.setCapability("proxy", json);*/

          // JSON type 2.
          // So with the JSON object I get a httpProxyPort not a valid option....
          /*JsonObject json = new JsonObject();
          json.addProperty("proxyType", "manual");
          json.addProperty("httpProxy", "localhost");
          json.addProperty("httpProxyPort", 9091);
          //json.addProperty("ftpProxy", "localhost");
          //json.addProperty("ftpProxyPort", 9091);
          json.addProperty("sslProxy", "localhost");
          json.addProperty("sslProxyPort", 9091);
          capabilities.setCapability(CapabilityType.PROXY, json);*/

          profile.setAcceptUntrustedCertificates(true);
          profile.setAssumeUntrustedCertificateIssuer(false);
          profile.setPreference("broswer.tabs.remote.autostart.2", false);
          capabilities.setCapability(FirefoxDriver.PROFILE,profile);
          // Marionette is for older versions of firefox with geckodriver.  So if you want to do
          // That you need to have it enabled.

          // Currently I'm getting a few different results depending on what I have this set to.
          // TRUE or not set:
            // Session not created: InvalidArgumentError: Expected [object Undefined] undefined to be an integer
            // Opens browser.  Closes it too.  Fails right away.
          // FALSE:
            // java.lang.reflect.InvocationTargetException
            // Opens the browser and then doesn't do anything with it.  Times out.
          //capabilities.setCapability(FirefoxDriver.MARIONETTE,true);
    }
    return capabilities;
  }
}

【问题讨论】：

用profile.setPreference("network.proxy.http", "localhost");代替profile.setPreference("network.proxy.http", "127.0.0.1");看看是否有帮助
Tarun，感谢您的尝试……没有变化。仍然是相同的堆栈跟踪。
你能贴出你试过的完整代码吗？
唯一影响firefox的其他代码是：profile.setAcceptUntrustedCertificates(true); profile.setAssumeUntrustedCertificateIssuer(false); profile.setPreference("broswer.tabs.remote.autostart.2", false); capabilities.setCapability(FirefoxDriver.PROFILE,profile);
感觉初始化对象有错误，能不能把你的代码用对象初始化来展示一下？

标签： java selenium proxy

【解决方案1】：

在阅读论坛等几天后，发现 Firefox 57 版有帮助！它已在该版本中修复。使用 selenium-java 3.6.0, selenium firefox driver 3.6.0, with firefox 57 (目前为 beta), mac os x 10.12.6, geckodriver 0.19.0 解决了这个问题！

//call with String "host:port"
private FirefoxDriver createDriverWithProxy( String proxyIpPort) {

    org.openqa.selenium.Proxy proxy = new org.openqa.selenium.Proxy();
    proxy.setHttpProxy(proxyIpPort)
            .setFtpProxy(proxyIpPort)
            .setSslProxy(proxyIpPort);
    DesiredCapabilities cap = new DesiredCapabilities();
    cap.setCapability(CapabilityType.PROXY, proxy);

    FirefoxOptions options = new FirefoxOptions(cap);
    FirefoxDriver driver = new FirefoxDriver(options);
    return driver;
}

【讨论】：

如何将 noProxy 添加到功能中？
@Ismoh proxy.setNoProxy(java.lang.String noProxy) 在此处查看文档seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/…

【解决方案2】：

火狐：59.0.1
geckodriver-v0.20.0-win64
selenium-java-3.9.1.jar
java8

这很好用。

FirefoxOptions options = new FirefoxOptions();

options.addPreference("network.proxy.type", 1);
options.addPreference("network.proxy.socks", "23.23.23.1");
options.addPreference("network.proxy.socks_port", 9150);
options.addPreference("network.proxy.socks_remote_dns", true);

WebDriver driver = new FirefoxDriver(options);

【讨论】：