Python Selenium: Handling Webdriver exception

Question

I am using Beautfilsoup for extracting app related information from Google Play Store. I am extracting app name, overall rating, no of people rated the app, and reviews posted by the reviewers. However, when I run the program it throws webdriver exception For instance: when program tries to retrieve information from this app ( 'https://play.google.com/store/apps/details?id=com.tudasoft.android.BeMakeup&hl=en&showAllReviews=true'), it throws error. Here is my code:

import bs4 as bs
from selenium import webdriver 

driver = webdriver.Chrome(path)
driver.get(url)
soup = bs.BeautifulSoup(driver.page_source, 'html.parser')

I got error on third line and the start of error is:

---------------------------------------------------------------------------
WebDriverException                        Traceback (most recent call last)
<ipython-input-270-4e8a1ef443f2> in <module>()
----> 1 soup = bs.BeautifulSoup(driver.page_source, 'html.parser')

and end of error is:

(Session info: chrome=79.0.3945.88)
  (Driver info: chromedriver=2.41.578706 (5f725d1b4f0a4acbf5259df887244095596231db),platform=Mac OS X 10.15.2 x86_64)

I tried to used exception handling but didn't fix the issue Could anyone guide me how to overcome this kind of situation?

If you see your Chrome driver chromedriver=2.41.578706 and chrome versions chrome=79.0.3945.88 are incompatible.First update your chrome driver and try it again. — KunduK
– KunduK, Commented Jan 8, 2020 at 10:50
I think the the issue is with the emojis in the reviews. At first I was getting the errors, But When I go into Dev Tools and remove those emojies, it runs fine. Not sure how to go about that issue though programatically. I read this here — chitown88
– chitown88, Commented Jan 8, 2020 at 10:57

undetected Selenium · Accepted Answer · 2020-01-08 11:01:07Z

2

This error message...

WebDriverException                        Traceback (most recent call last)
.
(Session info: chrome=79.0.3945.88)
  (Driver info: chromedriver=2.41.578706 (5f725d1b4f0a4acbf5259df887244095596231db),platform=Mac OS X 10.15.2 x86_64)

...implies that the ChromeDriver was unable to communicate with the Browsing Context i.e. Chrome Browser session.

Your main issue is the incompatibility between the version of the binaries you are using as follows:

You are using chromedriver=41.0
Release Notes of chromedriver=2.41 clearly mentions the following :

Supports Chrome v67-69

You are using chrome=79.0
Release Notes of ChromeDriver v79.0 clearly mentions the following :

Supports Chrome version 79

So there is a clear mismatch between ChromeDriver v2.41 and the Chrome Browser v79.0

Solution

Ensure that:

Selenium is upgraded to current levels Version 3.141.59.
ChromeDriver is updated to current ChromeDriver v79.0.3945.36 level.
Chrome is updated to current Chrome Version 79.0 level. (as per ChromeDriver v79.0 release notes)
Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
(WindowsOS only) Use CCleaner tool to wipe off all the OS chores before and after the execution of your Test Suite.
(LinuxOS only) Free Up and Release the Unused/Cached Memory in Ubuntu/Linux Mint before and after the execution of your Test Suite.
If your base Web Client version is too old, then uninstall it and released version of Web Client.
Take a System Reboot.
Execute your @Test as non-root user.
Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.

answered Jan 8, 2020 at 11:01

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user2293224 Over a year ago

For the last point you mentioned, if I am retrieving multiple url (using for loop), do I need to use driver.quit() at the end of loop (before loading the new url)?

undetected Selenium Over a year ago

@user2293224 It's always advisable to start each test in a cleaner way, i.e. spinning up a new Browsing Context and in these cases you can easily driver.quit(). Having said that if your tests have dependency, you may have to make change to your Test Architecture to reuse the existing session. In this case you can't driver.quit(). Inshort significance of driver.quit() is to keep the memory usage lower.

user2293224 Over a year ago

I followed your instructions: updated the chrome driver, selenium, and restarted the system. When I ran the program, it throws same error but with some different details. Here is the end of the error message:

user2293224 Over a year ago

(Session info: chrome=79.0.3945.88), this time it didn't print version of webdriver

undetected Selenium Over a year ago

@user2293224 Great, so we have solved the error you mentioned with in your question. Shouldn't we close this discussion and discuss the current error within a new thread?

Collectives™ on Stack Overflow

Python Selenium: Handling Webdriver exception

1 Answer 1

Solution

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Solution

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related