How to execute a multithreaded selenium program in python?

Question

I'd like to execute the following code using a multithreaded solution

Can anyone suggest how to improve the solution?

from selenium import webdriver

with open('proxy.txt', 'r') as f: 
    for line in f:
        print ("Connected with IP: {}".format(line))
        PROXY = line # IP:PORT or HOST:PORT
        chrome_options = webdriver.ChromeOptions()
        chrome_options.add_argument('--proxy-server=http://%s' % PROXY)
        driver = webdriver.Chrome(chrome_options=chrome_options)
        driver.get("http://whatismyipaddress.com")
        driver.quit()

Please explain more detailed and more deeply what you are trying to archive. — Klaus D.
– Klaus D., Commented Jan 17, 2019 at 22:46
@KlausD. i use this code for visits but i need to make him doing this process on multi threads so it can work faster? i hope im clear — mark contira
– mark contira, Commented Jan 17, 2019 at 22:51

Alan Kavanagh · Accepted Answer · 2019-01-18 01:58:57Z

3

import os 
from multiprocessing import Pool

from selenium import webdriver

def check_ip(proxy):
    print ("Connected with IP: {}".format(proxy))
    options = webdriver.chrome.options.Options()
    options.add_argument('--proxy-server=http://{}'.format(proxy))
    driver = webdriver.Chrome(options=options)
    driver.get("http://whatismyipaddress.com")
    driver.quit()

if __name__ == '__main__':
    with open('./proxy.txt') as f:
        proxies = f.read().splitlines()
    with Pool() as pool:
        pool.map(check_ip, proxies)

Solution is based on the answer in this question

edited Jan 18, 2019 at 1:58

answered Jan 17, 2019 at 22:51

Alan Kavanagh

9,9338 gold badges46 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to execute a multithreaded selenium program in python?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related