Selenium AttributeError: 'WebDriver' object has no attribute 'find_element_by_css_selector'

Question

I am following this build of a scraper for LinkedIn job data.

Here is my code:

from selenium import webdriver
import time
import pandas as pd

url = 'https://www.linkedin.com/jobs/search?keywords=&location=San%20Francisco%2C%20California%2C%20United%20States&locationId=&geoId=102277331&f_TPR=&distance=100&position=1&pageNum=0'

wd = webdriver.Chrome(executable_path=r'/Users/voi/chromedriver')
wd.get(url)

no_of_jobs = int(wd.driver.find_element_by_css_selector('h1>span').get_attribute('innerText'))

I have seen this, and attempted the solution, but received a similar error, except with regards to the WebDriver object not having a driver attribute.

Here is the full error message:

cd /Users/voi ; /usr/bin/env /usr/local/bin/python3 /Users/voi/.vscode/extensions/ms-python.python-2
022.8.1/pythonFiles/lib/python/debugpy/launcher 59402 -- /Users/voi/jobscrape.py 
/Users/voi/jobscrape.py:7: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  wd = webdriver.Chrome(executable_path=r'/Users/voi/chromedriver')
Traceback (most recent call last):
  File "/Users/voi/jobscrape.py", line 10, in <module>
    no_of_jobs = int(wd.find_element_by_css_selector('h1>span').get_attribute('innerText'))
AttributeError: 'WebDriver' object has no attribute 'find_element_by_css_selector'

Dalija Prasnikar · Accepted Answer · 2024-05-09 13:02:09Z

39

Okay, I answered my own question. The individual methods find_element_by_* have been replaced by find_element, e.g.

no_of_jobs = int(wd.find_element(By.CSS_SELECTOR, 'h1>span'))

More info is here

edited May 9, 2024 at 13:02

Dalija Prasnikar♦

28.7k46 gold badges97 silver badges179 bronze badges

answered Jul 4, 2022 at 9:09

m. lekk

7811 gold badge5 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Akzy Over a year ago

You are also getting the DeprecationWarning to resolve this you can see this [stackoverflow.com/a/72275260/11863448] , hope this will help you.

Pikamander2 · Accepted Answer · 2022-09-22 14:26:58Z

Selenium recently removed the 16 deprecated find_element(s)_by_x functions in favor of a general find_element and find_elements function that take the "by" part as their first argument.

To update your code, you can use your IDE's find-and-replace-all feature to replace these 16 search terms:

.find_element_by_class_name(
.find_element(By.CLASS_NAME, 

.find_element_by_css_selector(
.find_element(By.CSS_SELECTOR, 

.find_element_by_id(
.find_element(By.ID, 

.find_element_by_link_text(
.find_element(By.LINK_TEXT, 

.find_element_by_name(
.find_element(By.NAME, 

.find_element_by_partial_link_text(
.find_element(By.PARTIAL_LINK_TEXT, 

.find_element_by_tag_name(
.find_element(By.TAG_NAME, 

.find_element_by_xpath(
.find_element(By.XPATH, 

.find_elements_by_class_name(
.find_elements(By.CLASS_NAME, 

.find_elements_by_css_selector(
.find_elements(By.CSS_SELECTOR, 

.find_elements_by_id(
.find_elements(By.ID, 

.find_elements_by_link_text(
.find_elements(By.LINK_TEXT, 

.find_elements_by_name(
.find_elements(By.NAME, 

.find_elements_by_partial_link_text(
.find_elements(By.PARTIAL_LINK_TEXT, 

.find_elements_by_tag_name(
.find_elements(By.TAG_NAME, 

.find_elements_by_xpath(
.find_elements(By.XPATH,

You'll also need to import By if you haven't already done so:

from selenium.webdriver.common.by import By

Michael Mintz · Accepted Answer · 2022-08-27 05:13:17Z

Selenium just removed that method in version 4.3.0. See the CHANGES: https://github.com/SeleniumHQ/selenium/blob/a4995e2c096239b42c373f26498a6c9bb4f2b3e7/py/CHANGES

Selenium 4.3.0
* Deprecated find_element_by_* and find_elements_by_* are now removed (#10712)
* Deprecated Opera support has been removed (#10630)
* Fully upgraded from python 2x to 3.7 syntax and features (#10647)
* Added a devtools version fallback mechanism to look for an older version when mismatch occurs (#10749)
* Better support for co-operative multi inheritance by utilising super() throughout
* Improved type hints throughout

You now need to use:

driver.find_element("css selector", SELECTOR)

In your example, you would use:

no_of_jobs = int(wd.find_element("css selector", "h1 > span").get_attribute("innerText"))

For improved reliability, you should consider using WebDriverWait in combination with visibility_of_element_located.

Here's how that might look:

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

# ...

element = WebDriverWait(wd, 10).until(
    EC.visibility_of_element_located(("css selector", "h1 > span"))
)
no_of_jobs = int(element.get_attribute("innerText"))

pxDav · Accepted Answer · 2023-01-16 16:27:02Z

Starting from Selenium version 4.0, the methods find_element_by_* and find_elements_by_* are no longer recommended to be used and have been replaced by the methods find_element() and find_elements().

These new methods allow you to locate elements on a web page by specifying a search strategy, such as by CSS selector, ID, name, etc.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

url = 'https://www.linkedin.com/jobs/search?keywords=&location=San%20Francisco%2C%20California%2C%20United%20States&locationId=&geoId=102277331&f_TPR=&distance=100&position=1&pageNum=0'

wd = webdriver.Chrome(executable_path=r'/Users/voi/chromedriver')
wd.get(url)
element = WebDriverWait(wd, 20).until(
    EC.visibility_of_element_located((By.CSS_SELECTOR, "h1>span"))
)

Please checkout details here

Yunnosch · Accepted Answer · 2022-11-22 06:43:27Z

0

To add on to the answer from @m.lekk ( https://stackoverflow.com/a/72854301/7733418 ), I also tried to use dir() to get all the attributes from the object and find the text attribute that contains the information that I need.

edited Nov 22, 2022 at 6:43

Yunnosch

26.8k9 gold badges46 silver badges66 bronze badges

answered Nov 22, 2022 at 1:21

Frank

1

2 Comments

Community Over a year ago

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

JW. Over a year ago

This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review

Collectives™ on Stack Overflow

Selenium AttributeError: 'WebDriver' object has no attribute 'find_element_by_css_selector'

5 Answers 5

1 Comment

Comments

Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

Comments

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related