Using a http proxy with headless firefox in Selenium webdriver in Python

Question

I'm using Firefox headless like this:

from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium import webdriver
import os
import sys

# Set the MOZ_HEADLESS environment variable which casues Firefox to
# start in headless mode.
os.environ['MOZ_HEADLESS'] = '1'

# Select your Firefox binary.
binary = FirefoxBinary('/usr/bin/firefox', log_file=sys.stdout)

# Start selenium with the configured binary.
driver = webdriver.Firefox(firefox_binary=binary)

But now I want to add a http proxy that requires a user/password. After searching around, I tried the following:

from selenium.webdriver.common.proxy import Proxy, ProxyType

myProxy = "xx.xx.xx.xx:80"

proxy = Proxy({
    'proxyType': ProxyType.MANUAL,
    'httpProxy': myProxy,
    'ftpProxy': myProxy,
    'sslProxy': myProxy,
    'noProxy': '' # set this value as desired
    })

driver = webdriver.Firefox(firefox_binary=binary, proxy=proxy)

I also tried

profile = webdriver.FirefoxProfile() 
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http", "xx.xx.xx.xx")
profile.set_preference("network.proxy.http_port", 80)
profile.update_preferences() 
driver=webdriver.Firefox(firefox_binary=binary,firefox_profile=profile)

Finally, I tried adding "socksUsername" and "socksPassword" with the creds to the proxy, more out of desperation than any real hope.

Needless to say none of these works, and testing shows requests are still using my usual IP, not the proxy.

Also system-wide proxy is not an option in this case.

Where should the http proxy credentials live? How can I use a proxy with headless firefox?

Testing

driver.get("https://www.ipinfo.io");
driver.find_element_by_xpath('//h4/following-sibling::p').text

hey is this issue already solve ? someone please post the answer — Haryono Sariputra
– Haryono Sariputra, Commented Feb 27, 2019 at 6:58

Reupiey · Accepted Answer · 2018-09-26 09:35:10Z

4

If your proxy requires a username and a password, you have to write it like that :

from selenium.webdriver.common.proxy import Proxy, ProxyType

myProxy = "username:password@proxyDomain:proxyPort"

proxy = Proxy({
    'proxyType': ProxyType.MANUAL,
    'httpProxy': myProxy,
    'ftpProxy': myProxy,
    'sslProxy': myProxy,
    'noProxy': '' # set this value as desired
    })

driver = webdriver.Firefox(firefox_binary=binary, proxy=proxy)

answered Sep 26, 2018 at 9:35

Reupiey

2771 silver badge9 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Eduard Florinescu · Accepted Answer · 2018-02-15 21:56:38Z

0

Have you try setting up a profile manually with

./firefox --ProfileManager

manually setup the proxy and then load the profile you manually set

from selenium import webdriver

url = "https://mail.google.com"
fp = webdriver.FirefoxProfile('/Users/<username>/Library/Application Support/Firefox/Profiles/71v1uczn.default')

driver = webdriver.Firefox(fp)

answered Feb 15, 2018 at 21:56

Eduard Florinescu

17.7k30 gold badges123 silver badges189 bronze badges

Comments

Samiie · Accepted Answer · 2018-02-20 14:52:41Z

0

You can try setting up an Environment variable "HTTP_PROXY" in following mnemonics:

http://<username>:<password>@<proxy_url>

Add your credentials separated by colon ':' before proxy url that is preceded by '@' for e.g.

http://username:[email protected]:8080/file.pac

answered Feb 20, 2018 at 14:52

Samiie

1244 bronze badges

Collectives™ on Stack Overflow

Using a http proxy with headless firefox in Selenium webdriver in Python

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related