Selenium WebDriverWait can't wait for execute_script twice?

Question

I tried to get the shadowRoot element in the page through execute_script(), I didn't want to use time.sleep(), so I chose WebDriverWait. When I called WebDriverWait.until() for the first time, it seemed to really wait; when I called WebDriverWait.until() for the second time, it didn’t seem to wait, and Chrome exited after calling maximize_window(). Finally, the following error is output.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as wait
from selenium.webdriver.common.keys import Keys


with webdriver.Chrome() as browser:
    wait = WebDriverWait(browser, 30, 1)
    
    browser.get("https://www.virustotal.com/gui/home/search")

    element = wait.until(
        lambda browser: browser.execute_script("""return document.querySelector('home-view').shadowRoot.querySelector('vt-ui-search-bar').shadowRoot.querySelector('vt-ui-text-input')""")
    )

    if element:
        element.send_keys('5f580868011b6c0deb8bde8355630019')
        element.send_keys(Keys.RETURN)
    
        browser.maximize_window()

        #Must be using time.sleep() ???
        
        result = wait.until(
            lambda browser: browser.execute_script("""return document.querySelector('file-view').shadowRoot.querySelector('vt-ui-file-card').shadowRoot.querySelector('vt-ui-generic-card').children[1].textContent.trim()""")
        )

        if result:
            print(result)

DevTools listening on ws://127.0.0.1:12247/devtools/browser/11eba3c5-3044-4835-94a9-8db6b56687f4
Traceback (most recent call last):
  File "d:\source\VsCode\py\jupyter\test15.py", line 22, in <module>
    result = wait.until(
  File "D:\Program Files\Python39\lib\site-packages\selenium\webdriver\support\wait.py", line 78, in until
    value = method(self._driver)
  File "d:\source\VsCode\py\jupyter\test15.py", line 23, in <lambda>
    lambda browser: browser.execute_script("""return document.querySelector('file-view').shadowRoot.querySelector('vt-ui-file-card').shadowRoot.querySelector('vt-ui-generic-card').children[1].textContent.trim()""")
  File "D:\Program Files\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 878, in execute_script
    return self.execute(command, {
  File "D:\Program Files\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "D:\Program Files\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.JavascriptException: Message: javascript error: Cannot read properties of null (reading 'shadowRoot')
  (Session info: chrome=97.0.4692.71)
Stacktrace:
Backtrace:
        Ordinal0 [0x008A6903+2517251]
        Ordinal0 [0x0083F8E1+2095329]
        Ordinal0 [0x00742848+1058888]
        Ordinal0 [0x00744F44+1068868]
        Ordinal0 [0x00744E0E+1068558]
        Ordinal0 [0x007456BA+1070778]
        Ordinal0 [0x007964F9+1402105]
        Ordinal0 [0x007864D3+1336531]
        Ordinal0 [0x00795BBF+1399743]
        Ordinal0 [0x0078639B+1336219]
        Ordinal0 [0x007627A7+1189799]
        Ordinal0 [0x00763609+1193481]
        GetHandleVerifier [0x00A35904+1577972]
        GetHandleVerifier [0x00AE0B97+2279047]
        GetHandleVerifier [0x00936D09+534521]
        GetHandleVerifier [0x00935DB9+530601]
        Ordinal0 [0x00844FF9+2117625]
        Ordinal0 [0x008498A8+2136232]
        Ordinal0 [0x008499E2+2136546]
        Ordinal0 [0x00853541+2176321]
        BaseThreadInitThunk [0x7683FA29+25]
        RtlGetAppContainerNamedObjectPath [0x76FB7A9E+286]
        RtlGetAppContainerNamedObjectPath [0x76FB7A6E+238]

Guy · Accepted Answer · 2022-01-06 05:57:04Z

1

By default until handles only TimeoutException, but you are getting JavascriptException on one of the attempts to get an attribute from shadowRoot that is not yet present and therefore is null. Until can handle null/None result, but only on the returned value, not while executing the condition method.

One solution (and the simpler) is adding JavascriptException to the ignored_exceptions list when creating the WebDriverWait instance

wait = WebDriverWait(browser, 30, 1, ignored_exceptions=JavascriptException)

Another solution will be to split the script to parts and check for the availability of shadowRoot each time

result = wait.until(
    lambda browser: browser.execute_script(
        """
        value = document.querySelector('file-view').shadowRoot.querySelector('vt-ui-file-card');
        if (value) {
            value = value.shadowRoot.querySelector('vt-ui-generic-card');
            if (value) {
                value = value.children[1].textContent.trim();
            }
        }
        return value;""")
)

answered Jan 6, 2022 at 5:57

Guy

51.2k10 gold badges49 silver badges96 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

cruisepandey Over a year ago

I ran this code and it seems to do the job. OP will have to make changes into querySelector to get the desired output, right now the output is db4ac82d2c8db0dbc73e772b0b02c9e3917bbf13363bc37a0001dd2851304a4a pythompy2 1.17 MB Size 2019-12-09 09:51:35 UTC

s f Over a year ago

Due to network reasons, both requests and responses are relatively slow. Should I wait for a while before calling execute_script() a second time? If I use time.sleep() to wait for the display, does wait.until() still make sense?

Guy Over a year ago

@fs you can always increase the time defined in WebDriverWait

Collectives™ on Stack Overflow

Selenium WebDriverWait can't wait for execute_script twice?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related