I've been trying to automate some tasks on a dynamically loaded page using Selenium, and it cannot locate dynamically generated elements by xpath, their value, tags or anything else, it's just like requesting page only fetches source code, without parsing scripts and generating additional content. As a web browser extension, it works really fine, but when exporting to Python module it doesn't. Is there any way to make it work in scripts as intended, just like it works in browser extension?
Say, we have a webpage like this:
<html>
(...)
<body>
<app-root></app-root>
<script src="REDACTED.js" defer></script><script src="REDACTED.js" nomodule defer></script><script src="REDACTED.js" defer></script><script src="REDACTED.js" defer></script><script src="REDACTED.js" defer></script></body>
</html>
And the browser parses these scripts fine, shows buttons, UI etc. Again, interacting with it using Selenium IDE browser extension does its job. When I am exporting it to Python script it looks like:
# Generated by Selenium IDE
import pytest
import time
import json
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
class TestTest1():
def setup_method(self, method):
self.driver = webdriver.Chrome()
self.vars = {}
def teardown_method(self, method):
self.driver.quit()
def test_test1(self):
self.driver.get("https://REDACTED")
# Logging in to the portal I am interacting with - works fine
logform = self.driver.find_element_by_id("loginForm")
logform.find_element_by_name("username").send_keys('REDACTED')
logform.find_element_by_name("password").send_keys('REDACTED')
logform.find_element_by_name("login").click()
# Below code does not do anything, as it's not able to find these elements
# on the webpage and interact with them
self.driver.find_element(By.CSS_SELECTOR, ".btn--cta").click()
self.driver.find_element(By.CSS_SELECTOR, ".create-flow-btn--major > img").click()
element = self.driver.find_element(By.CSS_SELECTOR, ".create-flow-btn--major > img")
actions = ActionChains(self.driver)
actions.move_to_element(element).perform()
element = self.driver.find_element(By.CSS_SELECTOR, "body")
actions = ActionChains(self.driver)
actions.move_to_element(element, 0, 0).perform()
self.driver.close()
p1 = TestTest1()
p1.setup_method(1)
p1.test_test1()
p1.teardown_method(1)
And after running it, the script stops trying to find requested element, and throwing the following exception:
(...) line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".btn--cta"}
Oh Mr. Wu, what shall I do?
enough waitto load page properly? If so, and you are still getting same error then please check if element is insideiframeorshadowroot element?WebDriverWait(browser, 10).until(EC.title_contains("SomeTitle"))to verify whether page loaded before, but didn't think that scripts might take a lot more time to load (which makes sense if you sit and think about it, but anyway). I've used now WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "something"))) and it finds the element, however another exception occurs later but from what I gather it's for another question. Problem solved thanks!