1

I am trying to scrape a website with selenium and I am using mostly xpath or CSS selector to grab elements. However I am noticing that these are dynamic (even though I read online that CSS selector shouldnt be) and I am having to re write the code often. I am fairly new to this and would like help figuring out what would be the best way to do this. Below is an example of an element that is an input box that I am trying to grab, I understand more definitive selectors like ID are more robust to use but I cant seem to find any identifiers in this case. Element below -

<dpm-input-number-bare><input size="1" type="text" placeholder="" class="ng-pristine ng-valid ng-touched"></dpm-input-number-bare>

This doesnt work -

driver.find_element_by_css_selector("ng-valid.ng-dirty.ng-touched")

Here is the higher level - its basically an input box to enter the fixed rate (the label right next to the box)

<div class="dpm-form-row"><dpm-input-number class="flex-6"><dpm-input-label><label>Fixed Rate</label></dpm-input-label><dpm-input-number-bare><input size="1" type="text" placeholder="" class="ng-pristine ng-valid ng-touched"></dpm-input-number-bare></dpm-input-number><div class="flex-6"></div></div>
Improve this question
5
  • How about dpm-input-number-bare > input ? You might have to go up more and find some text Commented Jul 23, 2020 at 23:53
  • Please post more of your HTML data. It will be needed to find a good anchor point. Commented Jul 24, 2020 at 0:00
  • You can try .find_element_by_css_selector("input.ng-pristine.ng-valid.ng-touched") Commented Jul 24, 2020 at 3:01
  • @frianH it didnt work :( Message: no such element: Unable to locate element: {"method":"css selector","selector":"input.ng-pristine.ng-valid.ng-touched"} Commented Jul 24, 2020 at 13:49
  • @E.Wiest added more details Commented Jul 24, 2020 at 13:53

4 Answers 4

Reset to default
1

You can locate the element with the following XPath :

driver.find_element_by_xpath('//input[@class="ng-pristine ng-valid ng-touched"][../preceding-sibling::dpm-input-label[1]/label[.="Fixed Rate"]]')

We use the label element as an anchor point. Get the input element which fulfill the following condition : the first preceding-sibling of its parent has a label child which contains the term "Fixed Rate".

If needed, add an expected condition (element can receive the click). Assuming you want to send "12" in the input box :

WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, '//input[@class="ng-pristine ng-valid ng-touched"][../preceding-sibling::dpm-input-label[1]/label[.="Fixed Rate"]]'))).send_keys("12")

Imports :

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

EDIT : Other XPath alternatives :

3 XPath using following-sibling axis :

//dpm-input-label[label[.="Fixed Rate"]]/following-sibling::dpm-input-number-bare[1]/input
//dpm-input-label[label[contains(.,"Fixed Rate")]]/following-sibling::dpm-input-number-bare[1]/input
//dpm-input-label[contains(.,"Fixed Rate")]/following-sibling::dpm-input-number-bare[1]/input

3 XPath using preceding-sibling axis and multiple contains for the input element :

//input[contains(@class,"ng-pristine") and contains(@class,"ng-valid") and contains(@class,"ng-touched")][../preceding-sibling::dpm-input-label[1]/label[.="Fixed Rate"]]
//input[contains(@class,"ng-pristine") and contains(@class,"ng-valid") and contains(@class,"ng-touched")][../preceding-sibling::dpm-input-label[1]/label[contains(.,"Fixed Rate")]]
//input[contains(@class,"ng-pristine") and contains(@class,"ng-valid") and contains(@class,"ng-touched")][../preceding-sibling::dpm-input-label[1][contains(.,"Fixed Rate")]]

4 XPath using preceding axis :

//input[@class="ng-pristine ng-valid ng-touched"][preceding::label[1][.="Fixed Rate"]]
//input[@class="ng-pristine ng-valid ng-touched"][preceding::label[1][contains(.,"Fixed Rate")]]
//input[contains(@class,"ng-pristine") and contains(@class,"ng-valid") and contains(@class,"ng-touched")][preceding::label[1][.="Fixed Rate"]]
//input[contains(@class,"ng-pristine") and contains(@class,"ng-valid") and contains(@class,"ng-touched")][preceding::label[1][contains(.,"Fixed Rate")]]
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Actually didnt work, but I am quite sure its just a syntax issue so repeating the DOM and the code below - <div class="dpm-form-row ng-star-inserted"> <dpm-input-number class="flex-6"> <dpm-input-label> <label>Fixed Rate</label> </dpm-input-label> <dpm-input-number-bare> <input size="1" type="text" placeholder="" class="ng-pristine ng-valid ng-touched"> </dpm-input-number-bare> </dpm-input-number> <div class="flex-6"> </div> </div>
XPath looks fine to me. Post has been edited with 10 other expressions. Post the relevant error code if needed.
0

Have you tried this?

driver.find_element_by_xpath('//input[@type="text"]')

However, if there are more than one elements by the same XPath, it might raise an error.

Improve this answer

1 Comment

I am afraid that doesnt work since there are multiple input boxes on the page :(
0

how about finding the element using its class

driver.find_element_by_class_name('ng-pristine ng-valid ng-touched')
Improve this answer

1 Comment

Message: invalid selector: Compound class names not permitted
0

Try following approach with xpath:

driver.find_element_by_xpath("//div[@class='dpm-form-row' and contains(., 'Fixed Rate')]//input[@class='ng-pristine ng-valid ng-touched']")
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.