22

I want to use selenium with a proxy which is password protected. The proxy is not fixed, but a variable. So this has to be done in the code (just setting up firefox on this particular machine to work with the proxy is less-than-ideal). So far I have the following code:

fp = webdriver.FirefoxProfile()
# Direct = 0, Manual = 1, PAC = 2, AUTODETECT = 4, SYSTEM = 5
fp.set_preference("network.proxy.type", 1)

fp.set_preference("network.proxy.http", PROXY_HOST)
fp.set_preference("network.proxy.http_port", PROXY_PORT)

driver = webdriver.Firefox(firefox_profile=fp)
driver.get("http://whatismyip.com")

At this point, the dialog pops up requesting the proxy user/pass.

Is there an easy way to either:

  1. Type in the user/pass in the dialog box.
  2. Provide the user/pass at an earlier stage.
4
  • 2
    hey mate, have you found solutions yet? Commented Mar 3, 2012 at 12:47
  • 1
    @Shane have you found solutions yet? Commented Sep 11, 2015 at 10:44
  • 2
    @ArsenIbragimov have you found solutions yet? Commented Oct 29, 2015 at 20:08
  • to use firefox with an extension, see this: stackoverflow.com/a/39903614/955422 Commented Oct 18, 2017 at 12:03

4 Answers 4

29

Selenium can't do that by itself. The only way I found helpful is described here. To be short, you need to add a browser extension on fly that does the authentication. It's much simpler than may seem to be.

Here is how it works for Chrome (in my case):

  1. Create a zip file proxy.zip containing two files:

background.js

var config = {
    mode: "fixed_servers",
    rules: {
      singleProxy: {
        scheme: "http",
        host: "YOU_PROXY_ADDRESS",
        port: parseInt(YOUR_PROXY_PORT)
      },
      bypassList: ["foobar.com"]
    }
  };

chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});

function callbackFn(details) {
    return {
        authCredentials: {
            username: "YOUR_PROXY_USERNAME",
            password: "YOUR_PROXY_PASSWORD"
        }
    };
}

chrome.webRequest.onAuthRequired.addListener(
        callbackFn,
        {urls: ["<all_urls>"]},
        ['blocking']
);

Don't forget to replace YOUR_PROXY_* to your settings.

manifest.json

{
    "version": "1.0.0",
    "manifest_version": 2,
    "name": "Chrome Proxy",
    "permissions": [
        "proxy",
        "tabs",
        "unlimitedStorage",
        "storage",
        "<all_urls>",
        "webRequest",
        "webRequestBlocking"
    ],
    "background": {
        "scripts": ["background.js"]
    },
    "minimum_chrome_version":"22.0.0"
}
  1. Add the created proxy.zip as an extension

    from selenium import webdriver
    from selenium.webdriver.chrome.options import Options
    
    chrome_options = Options()
    chrome_options.add_extension("proxy.zip")
    
    driver = webdriver.Chrome(executable_path='chromedriver.exe', chrome_options=chrome_options)
    driver.get("http://google.com")
    driver.close()
    

That's it. For me that worked like a charm. If you need to create proxy.zip dynamically or need PHP example then go to the original post

Sign up to request clarification or add additional context in comments.

7 Comments

Mike, it does not work with me - joxi.net/LmG3OVNTRbEMam The last line says WebDriverException: Message: unknown error: cannot process extension #1 from unknown error: cannot read manifest (Driver info: chromedriver=2.29.461591 (62ebf098771772160f391d75e589dc567915b233),platform=Windows NT 10.0.14393 x86_64)
so how to communicate with the addition extension? support i want to change the proxy dynamiclly
I've tried this for a SOCKS5 and changed scheme: "http", to scheme: "socks5" but no luck. Any ideas how this changes for socks5 instead of HTTP?
@IgorSavinkin Check the suffix of your manifest file -> .json and not .js
This solution doesn't work for me anymore, did something change? someone got an update on it?
|
4

Code worked for me

from selenium import webdriver

browser=webdriver.Firefox()

def login(browser):

    alert=browser.switch_to_alert()
    alert.send_keys("username"+webdriver.common.keys.Keys.TAB+"password")
    alert.accept() 

1 Comment

Ugly method, but the only one which worked for me, just a new way to switch to alert: driver.switch_to.alert.send_keys(user + TAB + pass) driver.switch_to.alert.accept() Important send all text together instead of call send_keys() three times.
0

Selenium 4 has a built-in solution for Basic Auth

// This "HasAuthentication" interface is the key!
HasAuthentication authentication (HasAuthentication) driver;

// You can either register something for all sites
authentication.register(() -> new UsernameAndPassword("admin", "admin"));

// Or use something different for specific sites
authentication.register(
  uri -> uri.getHost().contains("mysite.com"),

new UsernameAndPassword("AzureDiamond", "hunter2"));

https://www.selenium.dev/blog/2021/a-tour-of-4-authentication/

1 Comment

This code is not python. Does anyone know the python equivalent of it?
-8

Did you try PROXY_HOST = "http://username:[email protected]"?

Also:

Starting with Selenium 2.0 beta 1, there is built in support for handling popup dialog boxes.

3 Comments

PROXY_HOST = "username:[email protected]" is not working. As to popup dialog handling, how do i locate user/pass elements to input?
@Shane, did you try PROXY_HOST = "username:[email protected]" or PROXY_HOST = "http://username:[email protected]". The second should pass the basic auth. Regarding the elements location: if you can switch to the dialog, you always can navigate its elements sending tab key signal.
This works only for outdated versions of chrome and selenium

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.