Error while using Firefox headless, Selenium and Python

Question

I am trying to use firefox headless, Selenium framework and Python to fetch webpage on Amazon EC2 Ubuntu linux. My code looks like this:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

options = Options()
options.headless = True

driver = webdriver.Firefox(options=options,executable_path='/home/ubuntu/geckodriver')
driver.get("https://google.com")
print('Done')
driver.quit()

Now when I run this, I get the following error:

Traceback (most recent call last):
  File "test1.py", line 7, in <module>
    driver = webdriver.Firefox(options=options,executable_path='/home/ubuntu/geckodriver')
  File "/home/ubuntu/.local/lib/python3.8/site-packages/selenium/webdriver/firefox/webdriver.py", line 170, in __init__
    RemoteWebDriver.__init__(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Connection refused (os error 111)

I have made sure that my geckodriver and firefox versions are compatible, I have tried rebooting my EC2 instance but nothing is working.

Any help is appreciated.

Mojo Jojo · Accepted Answer · 2020-08-23 15:58:36Z

1

Update: It seems like it was an OS issue. When I created a new EC2 instance using Amazon Linux, same code is working without issue. The older EC2 instance (Ubuntu) is still giving me the same error.

answered Aug 23, 2020 at 15:58

Mojo Jojo

3663 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Manoj AP · Accepted Answer · 2020-08-09 14:02:41Z

0

try this [with webdriver-manager ]

pip install webdriver-manager 
from webdriver_manager.firefox import GeckoDriverManager
self.browser = webdriver.Firefox(executable_path=GeckoDriverManager().install())

and it will automatically fix any driver error you have

answered Aug 9, 2020 at 14:02

Manoj AP

116 bronze badges

1 Comment

Mojo Jojo Over a year ago

Thanks for replying but this did not solve the issue.

user10767584 · Accepted Answer · 2020-08-09 16:01:04Z

0

Could that be used.

from selenium.webdriver.firefox.options import Options

options = Options()
options.add_argument('--headless')
driver = webdriver.Firefox(executable_path='path to the driver', options=options)

answered Aug 9, 2020 at 16:01

user10767584

1 Comment

Mojo Jojo Over a year ago

Thanks for replying but this did not solve the issue.

Manoj AP · Accepted Answer · 2020-08-12 03:41:41Z

0

This is the complete working code, I tested it on Windows machine with Pycharm community edition IDE

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from webdriver_manager.firefox import GeckoDriverManager
options = Options()
options.headless = True
driver = webdriver.Firefox(executable_path=GeckoDriverManager().install(),firefox_options=options)
driver.get("https://google.com")
print('Done')
driver.quit()

answered Aug 12, 2020 at 3:41

Manoj AP

116 bronze badges

Comments

pkastner · Accepted Answer · 2022-04-19 03:04:52Z

0

Working attempt in 2022 with Service object:

from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
from webdriver_manager.firefox import GeckoDriverManager

options = Options()
options.add_argument('--headless')

driver = webdriver.Firefox(service=Service(GeckoDriverManager().install()),options=options)
driver.get("https://www.google.com")
print('Done')
driver.quit()

answered Apr 19, 2022 at 3:04

pkastner

331 silver badge9 bronze badges

Collectives™ on Stack Overflow

Error while using Firefox headless, Selenium and Python

5 Answers 5

Comments

1 Comment

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

1 Comment

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related