Click not working for selenium in web scraping using python

Question

I am working on web scraping using selenium(python). There is chunk on code which is really long. So , i am using loops. When i am running the code lines separately, it is working fine but when i am using the loop, its not working. Here are the two errors:

WebDriverException: Message: unknown error: Element is not clickable at point (862, 13). Other element would receive the click: <div class="skycom_container">...</div>
(Session info: chrome=46.0.2490.80)
(Driver info: chromedriver=2.20.353145 (343b531d31eeb933ec778dbcf7081628a1396067),platform=Windows NT 6.1 SP1 x86_64)

WebDriverException: Message: unknown error: Element is not clickable at point (924, 786). Other element would receive the click: <div id="silentUIblocker" style="display: block;"></div>
(Session info: chrome=46.0.2490.80)
(Driver info: chromedriver=2.20.353145 (343b531d31eeb933ec778dbcf7081628a1396067),platform=Windows NT 6.1 SP1 x86_64)

Are these common or specific errors?? This is happening just before using the click() statement.

Here is my code:

from selenium import webdriver
import time
driver = webdriver.Chrome('C:\Users\name\Downloads\chromedriver_win32 (3)\chromedriver.exe')


driver.get('https://www.sky.com/shop/beta?s_tnt=87085:31:0')
driver.find_element_by_xpath('//*[@id="app"]/div/div/div[2]/div/div[5]/article/button/div[1]/div[2]/div/h2').click()
driver.find_element_by_xpath('//*[@id="app"]/div/div/div[2]/div/div[6]/section/div/div/div/div/div[1]/article/a').click()
driver.find_element_by_xpath('//*[@id="polaris"]/div/div/div/section/div/div[2]/a[2]').click()
driver.find_element_by_xpath('//*[@id="dsl-postcode"]').send_keys("E11 2LX")
driver.find_element_by_xpath('//*[@id="dsl-check-landline"]').click()
driver.find_element_by_xpath('//*[@id="dsl-addresses"]/option[2]').click()
driver.find_element_by_xpath('//*[@id="dsl-multiple-address-select"]').click()
driver.find_element_by_xpath('//*[@id="dsl-numberPortingNo"]').click()
driver.find_element_by_xpath('//*[@id="dsl-number-porting-submit"]').click()
driver.find_element_by_xpath('//*[@id="summaryBackLink"]').click()
driver.find_element_by_xpath('//*[@id="oneOffCostToolTip"]').click()
bb_pack = ["SKY_FIBRE_CAPPED", "BB_MAX"]
for i in bb_pack:
  driver.find_element_by_xpath('//*[@id="productButtonControls_%s"]/label' % i).click()
  bb_name1.append(driver.find_element_by_xpath('//*[@id="productButtonControls_%s"]/label' % i).text)
  pack = ["ANYTIME_EXTRA", "INTERNATIONAL_EXTRA"]
  for j in pack:
    driver.find_element_by_xpath('//*[@id="productButtonControls_ST_%s"]/label' % j).click()
    bb_name2.append(driver.find_element_by_xpath('//*[@id="productButtonControls_ST_%s"]/label' % j).text)
    #more details in this loop

Consider upgrading your Chrome to 60.0 & chromedriver to 2.30. Thanks — undetected Selenium
– undetected Selenium, Commented Aug 3, 2017 at 8:59
Possible duplicate of Selenium Web Driver & Java. Element is not clickable at point (36, 72). Other element would receive the click: — undetected Selenium
– undetected Selenium, Commented Aug 3, 2017 at 9:01
This generally happens when Dom is getting updated and webdriver tries to click or another hidden element is blocking your element. Try giving wait/sleeps between the clicks. — Prakash Palnati
– Prakash Palnati, Commented Aug 3, 2017 at 9:01
@Prakash, i tried that, i have almost 400 such cases, applying sleep will take really long time. Is there any other option? — sky_bird
– sky_bird, Commented Aug 3, 2017 at 9:04

Shubham Jain · Accepted Answer · 2017-08-03 12:14:49Z

1

Use fluent wait as below. It will wait till your element is not ready on page:-

WebElement waitsss(WebDriver driver, By elementIdentifier){
     Wait<WebDriver> wait =
                new FluentWait<WebDriver>(driver).withTimeout(60, TimeUnit.SECONDS)
                                                 .pollingEvery(1, TimeUnit.SECONDS)
                                             .ignoring(NoSuchElementException.class);

return wait.until(new Function<WebDriver, WebElement>()
        {
            public WebElement apply(WebDriver driver) {
                   return driver.findElement(elementIdentifier);
            }
            });
}

The wait should work for you. If still the problem exists then use JavascriptExecutor . It will operate directly through JS. It should work. I am giving an example to click any element using JavascriptExecutor

WebElement element = driver.findElement(By.id("gbqfd"));
JavascriptExecutor executor = (JavascriptExecutor)driver;
executor.executeScript("arguments[0].click();", element);

I just saw you are using python, in python, it should be something like below :-

driver.execute_script("arguments[0].click();", element)

The script should be like below :-

driver.execute_script("document.getElementsByClassName('skycom_container')[0].click()")
driver.execute_script("document.getElementById('silentUIblocker').click()")

Final code

    driver = webdriver.Chrome('C:\Users\name\Downloads\chromedriver_win32 (3)\chromedriver.exe')
    driver.implicitly_wait(30) # seconds
    driver.get('https://www.sky.com/shop/beta?s_tnt=87085:31:0')
    driver.find_element_by_xpath('//*[@id="app"]/div/div/div[2]/div/div[5]/article/button/div[1]/div[2]/div/h2').click()
    element=driver.find_element_by_xpath('//*[@id="app"]/div/div/div[2]/div/div[6]/section/div/div/div/div/div[1]/article/a') 
    driver.execute_script("arguments[0].click();", element)
    driver.find_element_by_xpath('//*[@id="polaris"]/div/div/div/section/div/div[2]/a[2]').click()
    driver.find_element_by_xpath('//*[@id="dsl-postcode"]').send_keys("E11 2LX")
    driver.find_element_by_xpath('//*[@id="dsl-check-landline"]').click()
    driver.find_element_by_xpath('//*[@id="dsl-addresses"]/option[2]').click()
    driver.find_element_by_xpath('//*[@id="dsl-multiple-address-select"]').click()
    driver.find_element_by_xpath('//*[@id="dsl-numberPortingNo"]').click()
    driver.find_element_by_xpath('//*[@id="dsl-number-porting-submit"]').click()
    driver.find_element_by_xpath('//*[@id="summaryBackLink"]').click()
    driver.find_element_by_xpath('//*[@id="oneOffCostToolTip"]').click()

Hope it will help you :)

edited Aug 3, 2017 at 12:14

answered Aug 3, 2017 at 9:11

Shubham Jain

17.8k17 gold badges88 silver badges135 bronze badges

Sign up to request clarification or add additional context in comments.

11 Comments

sky_bird Over a year ago

Could you please tell me the exact code lines for python? Thanx in advance..:)

Shubham Jain Over a year ago

As per the below link if you have classname or id you can click directly .. refer :- stackoverflow.com/questions/36987006/…

sky_bird Over a year ago

Is it possible for you to go through my code and help me?

Shubham Jain Over a year ago

can you tell me the id or class of the element which causing this error. be sure that id or class must be unique on the DOM

sky_bird Over a year ago

<div class="skycom_container">...</div> and <div id="silentUIblocker" style="display: block;"></div>. I got this from the error generated.

|

Andersson · Accepted Answer · 2017-08-03 09:18:58Z

1

Try to add below imports

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

and insert below line just before making click():

WebDriverWait(driver, 10).until(EC.invisibility_of_element_located((By.ID, "silentUIblocker")))

This should allow you wait until element that intercept the click disappeared

answered Aug 3, 2017 at 9:18

Andersson

52.8k18 gold badges83 silver badges132 bronze badges

8 Comments

sky_bird Over a year ago

Yeah, it works. There is another error i just added. Could you see that too?

Andersson Over a year ago

Try to solve in the same way: WebDriverWait(driver, 10).until(EC.invisibility_of_element_located((By.CLASS_NAME, "skycom_container")))

sky_bird Over a year ago

Still not done..getting this error: C:\Anaconda2\lib\site-packages\selenium\webdriver\support\wait.pyc in until(self, method, message) 78 if time.time() > end_time: 79 break ---> 80 raise TimeoutException(message, screen, stacktrace) 81 82 def until_not(self, method, message=''): TimeoutException: Message:

sky_bird Over a year ago

Even the WebdriverWait has stopped working! Why is this even happening!!? I really need help..:(

Andersson Over a year ago

Can you share page URL and description of what you're trying to do?

|

Collectives™ on Stack Overflow

Click not working for selenium in web scraping using python

2 Answers 2

11 Comments

8 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

11 Comments

8 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related