Selecting values from drop down of website through selenium Python

Question

I am trying web scraping using Python selenium. I am getting the following error: Message: element not interactable (Session info: chrome=78.0.3904.108) I am trying to access option element by id, value or text. It is giving me this error. I am using Python-3. Can someone help to explain where am I going wrong. I used the select tag using xpath and also tried css_selector. The select tag is selected and I can get the output of select tag selected. Here is my code for a better understanding: Code-1:-

path = r'D:\chromedriver_win32\chromedriver.exe'
browser = webdriver.Chrome(executable_path = path)
website = browser.get("https://publicplansdata.org/resources/download-avs-cafrs/")
el = browser.find_element_by_xpath('//*[@id="ppd-download-state"]/select')
for option in el.find_elements_by_tag_name('option'):
  if option.text != None:
    option.click()
    break

Blockquote

Code-2:-

select_element = Select(browser.find_element_by_xpath('//*[@id="ppd-download-state"]/select'))
# this will print out strings available for selection on select_element, used in visible text below
print(o.value for o in select_element.options)
select_element.select_by_value('AK')

Both codes give the same error how can I select values from drop down from website

Same as question: Python selenium select element from drop down. Element Not Visible Exception But the error is different. Tried the methods in comments

try this el = browser.find_element_by_xpath("//div[@id='ppd-report-downloads']/div[@id='ppd-download-state']/select") — Ed Bangga
– Ed Bangga, Commented Dec 11, 2019 at 1:07
Have you tried clicking on whatever element contains the downward arrow to the right of the text State first? I'm guessing that would make your select element visible. Just checked your site, you can click on "//span[text()='State']" first to make your select dropdown visible. — user5386938
– user5386938, Commented Dec 11, 2019 at 1:07
Ahh! That page probably uses bootstrap or something similar. The select has style=display:none. The dropdown that shows the states are in a UL element: ul.selectBox-dropdown-menu.selectBox-options. — user5386938
– user5386938, Commented Dec 11, 2019 at 1:25

score 0 · Accepted Answer · 2019-12-12 22:48:55Z

0

State, Plan, and Year:

browser.find_element_by_xpath("//span[text()='State']").click()    
browser.find_element_by_xpath("//a[text()='West Virginia']").click()

time.sleep(2) # wait for Plan list to be populated

browser.find_element_by_xpath("//span[text()='Plan']").click()    
browser.find_element_by_xpath("//a[text()='West Virginia PERS']").click()

time.sleep(2) # wait for Year list to be populated

browser.find_element_by_xpath("//span[text()='Year']").click()    
browser.find_element_by_xpath("//a[text()='2007']").click()

Don't forget to import time

edited Dec 12, 2019 at 22:48

answered Dec 11, 2019 at 1:30

user5386938

Sign up to request clarification or add additional context in comments.

5 Comments

Himali Shelat Over a year ago

Thank you so much Justin. The code did work:). I wanted to ask, where did you find this xpath? is it shown somewhere in inspect or you just created the path for span selection? I still have to work on selecting other two options(Plan & year) so it will be helpful.

user5386938 Over a year ago

It's shown in inspect. I've had experience crawling pages that use bootstrap so when I saw that the select was hidden, I went looking for the UL that bootstrap creates and displays.

Himali Shelat Over a year ago

One more doubt, After I set the value of any one state, in the state section, I want to select each plan which are loaded after we select the state. How can I access the Plan and also after selecting a plan, how can I select a particular year. @Justin. Thank you so much so far

user5386938 Over a year ago

@HimaliShelat, added clicks for Plan and Year

Himali Shelat Over a year ago

Thank you so much, It helped.

Collectives™ on Stack Overflow

Selecting values from drop down of website through selenium Python

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related