Python script runs in console but errors out as a script

Question

I'm working on a script to pull some information from a site that I must login to use. I'm using Python 2.7.12 and Selenium 3.4.3.

#!/usr/bin/python
from selenium import webdriver
browser = webdriver.Firefox(firefox_binary='/usr/bin/firefox', executable_path="./geckodriver")

# Get to the login page
browser.get('https://example.com')
browser.find_element_by_link_text('Application').click()

# Login
browser.find_element_by_id('username').send_keys('notmyusername')
browser.find_element_by_id('password').send_keys('notmypassword')
browser.find_element_by_css_selector('.btn').click()

# Open the application
browser.find_element_by_id('ctl00_Banner1_ModuleList_ctl01_lnkModule').click()

If I copy this code and paste it into the python console, it runs just fine and goes to the page I want. However, when I run the script from the terminal (bash on Linux Mint 18), it errors out. Here's the output with the try and catch statements removed:

Traceback (most recent call last):
  File "./script.py", line 14, in <module>
    browser.find_element_by_id('ctl00_Banner1_ModuleList_ctl01_lnkModule').click()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 289, in find_element_by_id
    return self.find_element(by=By.ID, value=id_)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 791, in find_element
    'value': value})['value']
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="ctl00_Banner1_ModuleList_ctl01_lnkModule"]

I don't even know how to go about troubleshooting this. Any help?

@muraliselenium Do you mean putting a wait in the script? Selenium appears to wait until the page is loaded before taking any action. The script errors out and stops running after it can't find the element by id. — mister-sir
– mister-sir, Commented Jul 23, 2017 at 15:50
yes u r right. but some times thread.sleep will helps in java. so just to know are you tried or not. — murali selenium
– murali selenium, Commented Jul 23, 2017 at 15:53
@muraliselenium You're totally correct! That fixed it. Thanks, and have an awesome day! — mister-sir
– mister-sir, Commented Jul 23, 2017 at 15:58

Thundzz · Accepted Answer · 2017-07-23 15:53:45Z

2

What is most probably happening is that when you run the script from bash, the script runs too quickly and the get_by_id operation is started before the browser has finished loading the page, which results in this error.

As @murali-selenium suggested, you should probably add some wait time before starting to look for stuff in the document.

That can be achived this way:

#!/usr/bin/python
from selenium import webdriver
import time

wait_time_sec = 1

browser = webdriver.Firefox(firefox_binary='/usr/bin/firefox', executable_path="./geckodriver")

# Get to the login page
browser.get('https://example.com')
time.sleep(wait_time_sec)
browser.find_element_by_link_text('Application').click()
time.sleep(wait_time_sec)

# Login
browser.find_element_by_id('username').send_keys('notmyusername')
browser.find_element_by_id('password').send_keys('notmypassword')
browser.find_element_by_css_selector('.btn').click()
time.sleep(wait_time_sec)

# Open the application
try:
    browser.find_element_by_id('ctl00_Banner1_ModuleList_ctl01_lnkModule').click()
except:
    print('failed')

#browser.stop()

answered Jul 23, 2017 at 15:53

Thundzz

7152 gold badges12 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

mister-sir Over a year ago

Per the selenium docs, this might be an even better solution, depending on the situation browser.implicitly_wait(10) # seconds. This makes it continue to check for up to 10 seconds before erroring out. I haven't tested it yet, though. Link to the documentation

mister-sir Over a year ago

Can confirm, browser.implicitly_wait(10) # seconds is definitely the better way to do this, in general.

mister-sir · Accepted Answer · 2021-10-23 00:44:24Z

1

Per selenium docs, setting browser.implicity_wait(10) # seconds tells the browser to poll the page for 10 seconds before deciding the element isn't there. By default it is set to 0 seconds. Once it is set, the setting persists for the life of the webdriver object.

There are a lot of other nifty tools to wait for elements to load, documented at readthedocs.io.

answered Oct 23, 2021 at 0:44

mister-sir

13510 bronze badges

Collectives™ on Stack Overflow

Python script runs in console but errors out as a script

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related