My python script has the headless chrome in selenium up and functional but how, if possible, can I use a proxy as well? How can I pass the proxy host port to my headless chrome browser?
options = webdriver.ChromeOptions()
options.add_argument('headless')
browser = webdriver.Chrome(chrome_options=options)
How do I use the proxy host port with selenium and mainly headless chrome? Thanks!
Something like this:
from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy, ProxyType
options = webdriver.ChromeOptions()
options.add_argument('headless')
desired_caps = options.to_capabilities()
prox = Proxy()
prox.proxy_type = ProxyType.MANUAL
prox.http_proxy = "ip_addr:port"
prox.socks_proxy = "ip_addr:port"
prox.ssl_proxy = "ip_addr:port"
prox.add_to_capabilities(desired_caps)
browser = webdriver.Chrome(desired_capabilities=desired_caps)
The simplest way to do is something like below. Put any ip address in the proxy variable to find out how it works. ip and port are separated by :.
from selenium import webdriver
proxy = "94.122.251.105:3128" #test with any ip address which supports `http` as well because the link within the script are of `http`
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--proxy-server={}'.format(proxy))
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get('http://www.lagado.com/proxy-test')
items = driver.find_element_by_css_selector(".main-panel p:nth-of-type(2)").text
print(items) #it should print out the ip address you are using in the `proxy` variable above
driver.quit()
