Selenium - identifying web element

Question

I am using Python to scrape data from a website. While I have been able to use Selenium to log in, I cannot identify the search field once logged in. It appears the web page loads with frames (not iframes), but I cannot access the frame with the search field.

I have tried changing the frame to the relevant frame (which seems to work - no error is thrown up) but then if I try searching for the search element by CSS / Xpath / Name / id I get a NoSuchElementException. I am using the Chrome webdriver.

Any suggestions? The page source is as follows:

  <html>
  <head>
    <title> XYZ </title>
    <meta http-equiv="content-type" content="text/html; charset=utf-8" />
    <meta http-equiv="Content-Script-Type" content="text/javascript" />
    <meta http-equiv="Content-Style-Type" content="text/css" />
    <meta http-equiv="content-language" content="en" />
    <script type="text/javascript">
    if (navigator && navigator.appVersion && navigator.appVersion.match("Safari") && !navigator.appVersion.match("Chrome")) {
        // hack to force a window redraw
        window.onload = function() {
        document.getElementsByTagName('html')[0].style.backgroundColor = '#000000';
        }
    }
    </script>
  </head>

  <frameset id="wc-frameset" rows="82,*" frameborder="no" border="0" framespacing="0">
    <frame frameborder="0" src="/frontend/header/" name="top" marginwidth="0" marginheight="0" scrolling="no" noresize="noresize" />
      <frameset cols="*,156,850,*" frameborder="NO" border="0" framespacing="0">
        <frame frameborder="0" src="/frontend/fillbar/" name="fillbar" marginwidth="0" marginheight="0" scrolling="no"  noresize="noresize" />
        <frame frameborder="0" src="/frontend/navigation/" name="navigation" marginwidth="0" marginheight="0" scrolling="no"  noresize="noresize" />
        <frame frameborder="0" src="/frontend/frames/" name="content_area" marginwidth="0" marginheight="0" scrolling="no" noresize>
        <frame frameborder="0" src="/frontend/fillbar/" name="fillbar" marginwidth="0" marginheight="0" scrolling="no"  noresize="noresize" />
      </frameset>
  </frameset>
</html>

The code that I have so far is:

username = driver.find_element_by_id("username")
password = driver.find_element_by_id("password")
username.send_keys("****")
password.send_keys("****")   
driver.find_element_by_class_name("bg-left").click()
#this bit works

driver.switch_to_frame("content_area")
#this seems to work too, got the frame name from the page source

search = driver.find_element_by_id("field-name")
search.send_keys("TEST")
#this fails, no element found

The target frame source code is:

      <div id="field-name" class="field field-StringField">
  <label for="name">Name</label>            <div class="input-con"><input id="name" name="name" type="text" value=""></div>
      </div>

Am I wrong? When you search for the id 'field-name' you get a div element but you want to send keystrokes to an input element. Wouldn't that be the element with id 'name'? — Bill Bell
– Bill Bell, Commented Apr 17, 2017 at 18:57
I've tried it that way as well, but still got an exception:selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"name"} — Owais Arshad
– Owais Arshad, Commented Apr 17, 2017 at 19:01
You mean that name = driver.find_element_by_id("name") raised an exception? — Bill Bell
– Bill Bell, Commented Apr 17, 2017 at 19:06
Yes using 'name' rather than 'field-name' still causes an exception — Owais Arshad
– Owais Arshad, Commented Apr 17, 2017 at 19:08

StrikerVillain · Accepted Answer · 2017-04-18 02:39:05Z

1

It is possible that there are duplicate elements in the page. Try the following in chrome:

Open url in chrome
Open developer tools F12
Press ESC to open the chrome console
Select your frame
Search for similar elements using xpath in console $x("//input[@id='name']")

This should list the number of elements.

answered Apr 18, 2017 at 2:39

StrikerVillain

3,8262 gold badges26 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Owais Arshad Over a year ago

Thanks! This helped. The frame had yet another frame within it, which I could only detect using the console. It appears to be working now. I had to switch to the content_area frame and then switch to the "main" frame within that frame!

Jayesh Doolani · Accepted Answer · 2017-04-17 20:59:38Z

1

Maybe you need to wait for the page to load up completely before continuing searching the element. You can try something like:

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait

driver.switch_to_frame("content_area")

try:
    # this line adds wait for the element to be visible 
    WebDriverWait(driver, 10).until(EC.visibility_of_element_located(By.ID, 'name'))
except TimeoutException:
    # display page timed out error

search = driver.find_element_by_id("name")
search.send_keys("TEST")

edited Apr 17, 2017 at 20:59

answered Apr 17, 2017 at 20:42

Jayesh Doolani

1,23310 silver badges13 bronze badges

Collectives™ on Stack Overflow

Selenium - identifying web element

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related