3

I'm looking for a way to make proxies work with authentication and headless. I tried this:

import os
import zipfile

PROXY_HOST = 'ooooo.com'  # rotating proxy or host
PROXY_PORT = 12345 # port
PROXY_USER = 'User_Proxy' # username
PROXY_PASS = 'Password:proxy' # password


manifest_json = """
{
    "version": "1.0.0",
    "manifest_version": 2,
    "name": "Chrome Proxy",
    "permissions": [
        "proxy",
        "tabs",
        "unlimitedStorage",
        "storage",
        "<all_urls>",
        "webRequest",
        "webRequestBlocking"
    ],
    "background": {
        "scripts": ["background.js"]
    },
    "minimum_chrome_version":"22.0.0"
}
"""

background_js = """
var config = {
        mode: "fixed_servers",
        rules: {
        singleProxy: {
            scheme: "http",
            host: "%s",
            port: parseInt(%s)
        },
        bypassList: ["localhost"]
        }
    };

chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});

function callbackFn(details) {
    return {
        authCredentials: {
            username: "%s",
            password: "%s"
        }
    };
}

chrome.webRequest.onAuthRequired.addListener(
            callbackFn,
            {urls: ["<all_urls>"]},
            ['blocking']
);
""" % (PROXY_HOST, PROXY_PORT, PROXY_USER, PROXY_PASS)


def get_chromedriver(use_proxy=False, user_agent=None):
    path = os.path.dirname(os.path.abspath(__file__))
    chrome_options = webdriver.ChromeOptions()
    #chrome_options = webdriver.Chrome
    if use_proxy:
        pluginfile = 'proxy_auth_plugin.zip'

        with zipfile.ZipFile(pluginfile, 'w') as zp:
            zp.writestr("manifest.json", manifest_json)
            zp.writestr("background.js", background_js)
            #chrome_options.add_argument("--window-size=1920,1080")
            chrome_options.add_argument('--headless')
        chrome_options.add_extension(pluginfile)
    if user_agent:
        chrome_options.add_argument('--user-agent=%s' % user_agent)
    driver = webdriver.Chrome(os.path.join(path, 'chromedriver'), 
             chrome_options=chrome_options)
    driver.get("https://ita.privateinternetaccess.com/pages/whats-my-ip/")

    print(driver.page_source)

driver = get_chromedriver(use_proxy=True)

But it gives an error:

selenium.common.exceptions.WebDriverException: Message: unknown error: failed to wait for extension background page to load: chrome-extension://fboilabmbpogaekkclhheepnkjcfifdn/_generated_background_page.html from unknown error: page could not be found: chrome-extension://fboilabmbpogaekkclhheepnkjcfifdn/_generated_background_page.html

1 Answer 1

2

You are basically trying to add a browser extension whilst in headless mode. Which sadly is not supported. If you found a solution to this please post the answer here. I am sure a lot of people are having the same issue (including me)

Sign up to request clarification or add additional context in comments.

1 Comment

You can use a virtual display instead of using headless mode. Example: gist.github.com/cgoldberg/4151516

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.