I am using Firefox WebDriver in Python 2.7 with Selenium. My python program starts Firefox browser and visits different websites when I run the program. But, I need to set the proxy with authentication, so that when program visits any website, it will visit through the proxy server.
There are some similar questions on SO. But, there is no specific solution for Selenium Firefox WebDriver of Python.
Dup of: How to set proxy AUTHENTICATION username:password using Python/Selenium
Selenium-wire: https://github.com/wkeeling/selenium-wire
Install selenium-wire
pip install selenium-wire
Import it
from seleniumwire import webdriver
Auth to proxy
options = {
'proxy': {
'http': 'http://username:password@host:port',
'https': 'https://username:password@host:port',
'no_proxy': 'localhost,127.0.0.1,dev_server:8080'
}
}
driver = webdriver.Firefox(seleniumwire_options=options)
Warning
Take a look to the selenium-wire cache folder. I had a problem because it take all my disk space. You have to remove it sometimes in your script when you want.
In addition to running Firefox with a profile which has the credentials saved. You can do it loading an extension that writes in the loginTextbox and password1Textbox of chrome://global/content/commonDialog.xul (the alert window).
There are already some extensions that will do the job. For instance: Close Proxy Authentication
from selenium import webdriver
from base64 import b64encode
proxy = {'host': HOST, 'port': PORT, 'usr': USER, 'pwd': PASSWD}
fp = webdriver.FirefoxProfile()
fp.add_extension('closeproxy.xpi')
fp.set_preference('network.proxy.type', 1)
fp.set_preference('network.proxy.http', proxy['host'])
fp.set_preference('network.proxy.http_port', int(proxy['port']))
# ... ssl, socks, ftp ...
fp.set_preference('network.proxy.no_proxies_on', 'localhost, 127.0.0.1')
credentials = '{usr}:{pwd}'.format(**proxy)
credentials = b64encode(credentials.encode('ascii')).decode('utf-8')
fp.set_preference('extensions.closeproxyauth.authtoken', credentials)
driver = webdriver.Firefox(fp)
chrome.storage not in browser's preferences (so you can't find them in about:config). I'd install some new WebExtension which does the job, set their proxy preferences, inspect its storage (chrome.storage.local.get((res)=>{console.log(res)})) and then load it dynamically in the geckodriver and set its preferences through execute_script.
You can write own firefox extension for proxy, and launch from selenium. You need write 2 files and pack it.
background.js
var proxy_host = "YOUR_PROXY_HOST";
var proxy_port = YOUR_PROXY_PORT;
var config = {
mode: "fixed_servers",
rules: {
singleProxy: {
scheme: "http",
host: proxy_host,
port: proxy_port
},
bypassList: []
}
};
function proxyRequest(request_data) {
return {
type: "http",
host: proxy_host,
port: proxy_port
};
}
browser.proxy.settings.set({value: config, scope: "regular"}, function() {;});
function callbackFn(details) {
return {
authCredentials: {
username: "YOUR_USERNAME",
password: "YOUR_PASSWORD"
}
};
}
browser.webRequest.onAuthRequired.addListener(
callbackFn,
{urls: ["<all_urls>"]},
['blocking']
);
browser.proxy.onRequest.addListener(proxyRequest, {urls: ["<all_urls>"]});
manifest.json
{
"name": "My Firefox Proxy",
"version": "1.0.0b",
"manifest_version": 2,
"permissions": [
"browsingData",
"proxy",
"storage",
"tabs",
"webRequest",
"webRequestBlocking",
"downloads",
"notifications",
"<all_urls>"
],
"background": {
"scripts": ["background.js"]
},
"browser_specific_settings": {
"gecko": {
"id": "[email protected]"
}
}
}
Next you need packed this files to zip archive in DEFLATED mode with .xpi at end like my_proxy_extension.xpi.
You have two choices:
OR
Run unsigned. For this step:
Open firefox flags at about:config and set options xpinstall.signatures.required to false
OR
Update firefox profile in:
Windows: C:\Program Files\Mozilla Firefox\defaults\pref\channel-prefs.js
Linux: /etc/firefox/syspref.js
Add next line to end of file:
pref("xpinstall.signatures.required",false);
After this steps run selenium and install this extension:
from selenium import webdriver
driver = webdriver.Firefox()
driver.install_addon("path/to/my_proxy_extension.xpi")
driver.get("https://yoursite.com")
browser with chrome in background.js. This code uses callbacks which is how chrome does it. Firefox uses promises. If you use chrome keyword it will work in compatibility mode.
browser with chrome. Thanks @Guerrilla
There is an example for Firefox + Python but without the authentication here. Then you can find other available parameters here in source code. So it looks like you need the following:
socksUsername
socksPassword
For example:
from selenium import webdriver
from selenium.webdriver.common.proxy import *
myProxy = "host:8080"
proxy = Proxy({
'proxyType': ProxyType.MANUAL,
'httpProxy': myProxy, # set this value as desired
'ftpProxy': myProxy, # set this value as desired
'sslProxy': myProxy, # set this value as desired
'noProxy': '' # set this value as desired
'socksUsername': = ''
'socksPassword': = ''
})
driver = webdriver.Firefox(proxy=proxy)
or with preferences:
driverPref = webdriver.FirefoxProfile()
driverPref.set_preference("network.proxy.type", 1)
.
.
.
driverPref.set_preference('network.proxy.socks', proxyHost)
driverPref.set_preference('network.proxy.socks_port', proxyPort)
driverPref.update_preferences()
driver = webdriver.Firefox(firefox_profile=driverPref)
EDIT:
I looked at it again and it seems that it is impossible to set authentication details in FF, even manually. The only way is just to remember the details that you have already entered which done by 2 parameters:
signon.autologin.proxy=true
network.websocket.enabled=false
that can be configured with the set_preference() method. You can also manually view all FF options by browsing to about:config.
In an addition to the answer with extension.
You can also use form filling to dynamically change credentials on your proxy. Just load the extension page, fill the form automatically and click save!
We can switch to authentication alert box and enter username password manually.
profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference('network.proxy.ssl_port', int(ProxyPort))
profile.set_preference('network.proxy.ssl', ProxyHost)
profile.set_preference("network.proxy.http", ProxyHost)
profile.set_preference("network.proxy.http_port", int(ProxyPort))
webdriver = webdriver.Firefox(firefox_profile=profile, executable_path=GECKO_DRIVER_PATH)
webdriver.get("https://whatismyipaddress.com/")
try:
alert = webdriver.switch_to_alert()
print("switched to alert window")
alert.send_keys(proxy_username + Keys.TAB + proxy_password)
alert.accept()
webdriver.switch_to.default_content()
except Exception:
print("Error in alert switch")
pass
