I need to use an authenticated proxy with my headless implementation of chrome. These are jobs that get executed on heroku dynos (if there's maybe a better way to implement a proxy directly on heroku dynos).
If I follow standard proxy implementations for selenium I get stuck at the authentication page where it asks for a username / password.
This is how I instantiate a headless chrome instance with watir & selenium:
def headless_browser
options = Selenium::WebDriver::Chrome::Options.new
# make a directory for chrome if it doesn't already exist
chrome_dir = File.join Dir.pwd, %w(tmp chrome)
FileUtils.mkdir_p chrome_dir
options.add_argument "--user-data-dir=#{chrome_dir}"
# set Random User Agent
options.add_argument "--user-agent=#{random_user_agent}"
# let Selenium know where to look for chrome if we have a hint from
# heroku. chromedriver-helper & chrome seem to work out of the box on osx,
# but not on heroku.
if chrome_bin = ENV["GOOGLE_CHROME_BIN"]
options.add_argument "no-sandbox"
options.binary = chrome_bin
# give a hint to webdriver here too
Selenium::WebDriver::Chrome.driver_path = \
'/app/vendor/bundle/bin/chromedriver'
end
options.add_argument '--allow-insecure-localhost'
# headless!
# keyboard entry wont work until chromedriver 2.31 is released
options.add_argument '--window-size=1200x600'
options.add_argument '--headless'
options.add_argument '--disable-gpu'
options.add_argument '--no-sandbox'
# instantiate the browser
browser = Watir::Browser.new :chrome, options: options
if Rails.env.development?
browser.goto "https://api.myip.com"
JSON.parse(Nokogiri::HTML.parse(browser.html).css('body').text)
end
end
Does anyone have any idea how to implement authenticated proxy usage with watir & selenium? I searched a lot and implemented a lot of different "solutions" but none worked for me.
- Tried setting a proxy via the selenium driver. If just left like this the authentication window pops up and I can't access that.
Selenium::WebDriver::Proxy.new(
http: '127.0.0.1:8080',
ssl: '127.0.0.1:8080'
)
I also tried the socks format: username:password@host:port
. Didn't work.
- I tried setting the
--proxy-server=
option to the driver. While it works with the plain proxy url, it behaves just like the above.