Looping through drop downs using selenium in python

Question

I am trying to simulated clicking through multiple options on an online data tool that ends with downloading an excel sheet given your filters.

I am currently using selenium and identifying xpaths.

I am able to get through a single iteration and get a single excel sheet, but I need to do it for every possible permutation of drop down choices. To do by hand is unrealistic, as there are thousands of options.

The website for context: https://data.cms.gov/mapping-medicare-disparities

Does anyone know of a function that can be done in selenium that will work?

My current strategy is to create lists with the xpaths and then try to do a permutation function to get all the combinations. However, this has not worked because the function: b.find_element_by_xpath only allows one xpath at a time.

examples of lists:

geography county state/territory

G1 = '//select[@id="geography"]//option[@value="c"]' G2 = '//select[@id="geography"]//option[@value="s"]'

Geo = [G1, G2]

creating pool of combinations

import itertools from itertools import product

for perm in product(Geo, Adjust, Analysis, Domain): print(perm)

actual code to use selenium

**from** selenium **import** webdriver
**from** selenium.webdriver.common.keys **import** Keys
b = webdriver.Firefox()

code to click through a popup

pop_up = b.find_element_by_xpath('/html/body/div[1]/button')
pop_up.click()

code trying to use xpath to select all options at once

b.find_element_by_xpath(('//select[@id="geography"]//option[@value="c"],
'//select[@id="adjust"]//option[@value="1"],'//select[@id="analysis"]
//option[@value="base"],'//select[@id="domain"]//option[@value="d1"]'))

error message: InvalidArgumentException: Message: invalid type: sequence, expected a string at line 1 column 28

This is because the find_element_by_xpath (I am assuming) will only look at 1 xpath at a time.

SuperStew · Accepted Answer · 2019-03-26 17:59:53Z

1

your syntax in code trying to use xpath... is wrong anyway, but you could just put all the xpaths in a list and loop through it.

xpathlist=['//select[@id="geography"]//option[@value="c"]', '//select[@id="adjust"]//option[@value="1"]',.....]

for xp in xpathlist:
    b.find_element_by_xpath(xp)
    #then add code to click or download or whatever

answered Mar 26, 2019 at 17:59

SuperStew

3,0742 gold badges18 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Looping through drop downs using selenium in python

examples of lists:

geography county state/territory

creating pool of combinations

actual code to use selenium

code to click through a popup

code trying to use xpath to select all options at once

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

examples of lists:

geography county state/territory

creating pool of combinations

actual code to use selenium

code to click through a popup

code trying to use xpath to select all options at once

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related