Click and view more pages with Selenium in Python

Question

Problem I need to go into all the top user profile in this page using Selenium. The Top users profile are located in the right of the page.

What i've done

    self.driver.get(response.url)
    user_list = self.driver.find_elements_by_xpath('//table[contains(@class,"W-100 Bc-c")]/tbody/tr')
    for single_user in user_list:
        single_user.find_element_by_xpath('.//td/a').click()
        time.sleep(3)

But I get this error message:

WebDriverException: Message: unknown error: Element is not clickable at point (865, 685). Other element would receive the click:

<div id="MouseoverMask" class="End-0 Start-0 T-0 B-0"></div>

Info Python 2.7.10 Selenium 2.48 Pycharm

EDIT+

I try to make a print of the name and it works:

   print(str( single_user.find_element_by_xpath('.//td/a').text ) )

But the click() no.

WebDriverException: Message: unknown error: Element is not clickable at point (865, 685). Other element would receive the click: <div id="MouseoverMask" class="End-0 Start-0 T-0 B-0"></div> — RedVelvet
– RedVelvet, Commented Oct 13, 2015 at 12:01
If you look at the HTML Selenium is seeing, it's completely blank (or it is for me) - so possibly Yahoo is taking measures against scraping. — user3468054
– user3468054, Commented Oct 13, 2015 at 12:12
@user3468054 mhhh i using selenium in order to take other things like url of the questions and it works. However if you look with debug the user_list will be created and the for can be executed for any single_user if you add a print("example") instead of the .click() — RedVelvet
– RedVelvet, Commented Oct 13, 2015 at 12:38

Hayha · Accepted Answer · 2015-10-13 13:20:03Z

1

if you sure that the object you get is the right one, often the problem is:

The object is not visible
Page was not fully loaded when you try to click on the object.

So just take a look on the Wait method provided by Selenium and be sure your object is visible

In order to wait an element to be clickable :

from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.ID,'someid')))

In your case, you can try to find each element with the id you got and click on it :

self.driver.get(response.url)
user_list = self.driver.find_elements_by_xpath('//table[contains(@class,"W-100 Bc-c")]/tbody/tr')
for single_user in user_list:
    id = single_user.find_element_by_xpath('.//td/a').get_attribute("id")
    self.driver.find_elements_by_id(id).click()
    time.sleep(3)

edited Oct 13, 2015 at 13:20

answered Oct 13, 2015 at 12:46

Hayha

2,2601 gold badge18 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

RedVelvet Over a year ago

Yes object is visibile and before i wait about 5 sec.

Hayha Over a year ago

Did you try to wait as the form i just added in my post?

RedVelvet Over a year ago

With wait = WebDriverWait(self.driver, 10) element = wait.until(EC.element_to_be_clickable((By.ID,'ya-leader-board')))

RedVelvet Over a year ago

Element is not clickable at point (871, 693). Other element would receive the click: <div id="MouseoverMask" class="End-0 Start-0 T-0 B-0"></div>

Hayha Over a year ago

Can you test my last edit (i'm not sure about syntax to get the id attribut)

Learner · Accepted Answer · 2015-10-13 13:16:37Z

0

I don't see any error at my end but after first click web elements are changed so you will not get the next web element as captured earlier in xpath. By the way try below code-

from selenium import webdriver
import time
driver = webdriver.Firefox()
driver.get('https://answers.yahoo.com/dir/index/discover?sid=396545663')
user_list = driver.find_elements_by_xpath('//table[contains(@class,"W-100 Bc-c")]/tbody/tr')
lnks = [i.find_element_by_xpath('.//td/a').get_attribute('href') for i in user_list]
for single_user in lnks:
    driver.get(single_user)
    time.sleep(3)

edited Oct 13, 2015 at 13:16

answered Oct 13, 2015 at 13:09

Learner

5,3001 gold badge29 silver badges39 bronze badges

Collectives™ on Stack Overflow

Click and view more pages with Selenium in Python

2 Answers 2

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related