1

I am creating a Linkedin job scraper in order of most recent, but I am finding it really difficult to target the 'Most recent' radio button as shown below.

So far, the 'Most relevant' menu is clicked on, but will not click on 'Most recent'. Help would be appreciated I can't seem to figure this one out :/

enter image description here

Code snippet

driver.get('https://uk.linkedin.com/jobs/bioinformatics-jobs?position=1&pageNum=0')
driver.implicitly_wait(10)
driver.find_element_by_class_name('collapsible-dropdown').click() # Clicks 'sort-by' menu.
driver.find_element_by_xpath('//*[@id="sortBy-0"]').click() # Error!

HTML

<button aria-expanded="true" aria-label="Sort By filter. Most relevant filter is currently applied. Clicking this button displays all Sort By filter options." class="filter-button filter-button--selected dropdown-to-modal__button collapsible-dropdown__button" data-tracking-control-name="public_jobs_sortBy" type="button">
        
        Most relevant
      <icon class="filter-button__icon"></icon>
</button>
<div class="collapsible-dropdown__list no-focus-ring" tabindex="-1">
<!-- -->
<fieldset class="filter-values-container">
<legend class="filter-values-container__legend sr-only">Sort By filter options</legend>
<div class="filter-values-container__filter-values">
<div class="filter-values-container__filter-value">
<input checked="" form="jserp-filters" id="sortBy-0" name="sortBy" type="radio" value="R"/>
<label for="sortBy-0">
        Most relevant
    </label>
</div>
<div class="filter-values-container__filter-value">
<input form="jserp-filters" id="sortBy-1" name="sortBy" type="radio" value="DD"/>
<label for="sortBy-1">
        Most recent
    </label>
</div>
</div>
</fieldset>
<button aria-label="Apply filters" class="filter__submit-button" data-tracking-control-name="public_jobs_sortBy" form="jserp-filters" type="submit">
    Done
  </button>
</div>

  <div class="filter-values-container__filter-value">
    <input id="sortBy-1" form="jserp-filters" name="sortBy" value="DD" type="radio">
    <label for="sortBy-1">
        Most recent
    </label>
  </div>

      </div>
    </fieldset>
  
  <button class="filter__submit-button" aria-label="Apply filters" form="jserp-filters" data-tracking-control-name="public_jobs_sortBy" type="submit">
    Done
  </button>

</div>

2 Answers 2

2

try this driver.find_element_by_xpath('//*[@id="jserp-filters"]/ul/li[1]/div/div/div/fieldset/div/div[2]').click() your xpath was incomplete

Sign up to request clarification or add additional context in comments.

Comments

1

Things to be noted down :

  1. You need Explicit waits.
  2. JS intervention.
  3. Reliable locators.
  4. Prefer CSS over Xpath.

Code :-

driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.get("https://uk.linkedin.com/jobs/bioinformatics-jobs?position=1&pageNum=0")
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.collapsible-dropdown>button[data-tracking-control-name='public_jobs_sortBy']"))).click()
driver.execute_script("arguments[0].click();", driver.find_element_by_css_selector("input[value='DD']"))

Imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

2 Comments

I always used xpath because it seemed easier, but now I know better, thanks!!!
No css performs far better in cross browser testing, also they play vital role in large automation test suite. But if you have do not have those requirements, xpath can be used.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.