0

I tried many of the web solution but I can't find how to find the HREF.

One of my code:

driver = webdriver.Chrome(executable_path='C:\chromedriver.exe')
driver.get(("https://cordis.europa.eu/search/fr?q=contenttype%3D%27project%27%20AND%20programme%2Fcode%3D%27H2020%27&p=1&num=100&srt=/project/contentUpdateDate:decreasing"))

time.sleep(20)
driver.execute_script("window.scrollTo(0,10000)")

elems = driver.find_elements(By.CSS_SELECTOR, ".c-card-search__title.ng-star-inserted [href]")
links = [elem.get_attribute('href') for elem in elems]

The web site: https://cordis.europa.eu/search/fr?q=contenttype%3D%27project%27%20AND%20programme%2Fcode%3D%27H2020%27&p=1&num=10&srt=/project/contentUpdateDate:decreasing

The insepctor:

<a _ngcontent-vtw-c99="" class="c-card-search__title ng-star-inserted" href="/project/id/881603"> Graphene Flagship Core Project 3 </a>
Improve this question

2 Answers 2

Reset to default
1

The href attributes are within the following <a> nodes.

<a _ngcontent-jdx-c99="" class="c-card-search__title ng-star-inserted" href="/project/id/881603"> Graphene Flagship Core Project 3 </a>

Solution

To print the values of the href attribute you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following locator strategies:

  • Using CSS_SELECTOR:

    driver.get("https://cordis.europa.eu/search/fr?q=contenttype%3D%27project%27%20AND%20programme%2Fcode%3D%27H2020%27&p=1&num=100&srt=/project/contentUpdateDate:decreasing")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='o-btn o-btn--large o-btn--light co-cookie-consent__button co-cookie-consent__button--allow']"))).click()
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.c-card-search__title.ng-star-inserted[href]")))])

  • Using XPATH:

    driver.get("https://cordis.europa.eu/search/fr?q=contenttype%3D%27project%27%20AND%20programme%2Fcode%3D%27H2020%27&p=1&num=100&srt=/project/contentUpdateDate:decreasing")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='o-btn o-btn--large o-btn--light co-cookie-consent__button co-cookie-consent__button--allow']"))).click()
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[@class='c-card-search__title ng-star-inserted' and @href]")))])

  • Console Output:

    ['https://cordis.europa.eu/project/id/881603', 'https://cordis.europa.eu/project/id/945539', 'https://cordis.europa.eu/project/id/964827', 'https://cordis.europa.eu/project/id/956396', 'https://cordis.europa.eu/project/id/956544', 'https://cordis.europa.eu/project/id/101037031', 'https://cordis.europa.eu/project/id/101025100', 'https://cordis.europa.eu/project/id/101007165', 'https://cordis.europa.eu/project/id/959234', 'https://cordis.europa.eu/project/id/952520', 'https://cordis.europa.eu/project/id/959879', 'https://cordis.europa.eu/project/id/101005177', 'https://cordis.europa.eu/project/id/101004714', 'https://cordis.europa.eu/project/id/946044', 'https://cordis.europa.eu/project/id/961989', 'https://cordis.europa.eu/project/id/962483', 'https://cordis.europa.eu/project/id/862011', 'https://cordis.europa.eu/project/id/891402', 'https://cordis.europa.eu/project/id/886365', 'https://cordis.europa.eu/project/id/870378', 'https://cordis.europa.eu/project/id/870743', 'https://cordis.europa.eu/project/id/834195', 'https://cordis.europa.eu/project/id/852985', 'https://cordis.europa.eu/project/id/814837', 'https://cordis.europa.eu/project/id/861278', 'https://cordis.europa.eu/project/id/842243', 'https://cordis.europa.eu/project/id/833507', 'https://cordis.europa.eu/project/id/825496', 'https://cordis.europa.eu/project/id/820350', 'https://cordis.europa.eu/project/id/820445', 'https://cordis.europa.eu/project/id/786890', 'https://cordis.europa.eu/project/id/824295', 'https://cordis.europa.eu/project/id/769086', 'https://cordis.europa.eu/project/id/768960', 'https://cordis.europa.eu/project/id/772337', 'https://cordis.europa.eu/project/id/793965', 'https://cordis.europa.eu/project/id/797942', 'https://cordis.europa.eu/project/id/757184', 'https://cordis.europa.eu/project/id/639020', 'https://cordis.europa.eu/project/id/694665', 'https://cordis.europa.eu/project/id/679403', 'https://cordis.europa.eu/project/id/677061', 'https://cordis.europa.eu/project/id/677493', 'https://cordis.europa.eu/project/id/679843', 'https://cordis.europa.eu/project/id/677823', 'https://cordis.europa.eu/project/id/716472', 'https://cordis.europa.eu/project/id/721537', 'https://cordis.europa.eu/project/id/725627', 'https://cordis.europa.eu/project/id/694343', 'https://cordis.europa.eu/project/id/677650', 'https://cordis.europa.eu/project/id/678423', 'https://cordis.europa.eu/project/id/679243', 'https://cordis.europa.eu/project/id/846617', 'https://cordis.europa.eu/project/id/101002761', 'https://cordis.europa.eu/project/id/848325', 'https://cordis.europa.eu/project/id/850488', 'https://cordis.europa.eu/project/id/854931', 'https://cordis.europa.eu/project/id/101027737', 'https://cordis.europa.eu/project/id/101030066', 'https://cordis.europa.eu/project/id/101020016', 'https://cordis.europa.eu/project/id/101024758', 'https://cordis.europa.eu/project/id/850769', 'https://cordis.europa.eu/project/id/863227', 'https://cordis.europa.eu/project/id/861910', 'https://cordis.europa.eu/project/id/878074', 'https://cordis.europa.eu/project/id/875905', 'https://cordis.europa.eu/project/id/862111', 'https://cordis.europa.eu/project/id/846793', 'https://cordis.europa.eu/project/id/862311', 'https://cordis.europa.eu/project/id/101003472', 'https://cordis.europa.eu/project/id/101036910', 'https://cordis.europa.eu/project/id/101036449', 'https://cordis.europa.eu/project/id/101008571', 'https://cordis.europa.eu/project/id/101018420', 'https://cordis.europa.eu/project/id/101037564', 'https://cordis.europa.eu/project/id/101000622', 'https://cordis.europa.eu/project/id/101021936', 'https://cordis.europa.eu/project/id/101037424', 'https://cordis.europa.eu/project/id/101008233', 'https://cordis.europa.eu/project/id/101003470', 'https://cordis.europa.eu/project/id/101008297', 'https://cordis.europa.eu/project/id/101003815', 'https://cordis.europa.eu/project/id/956355', 'https://cordis.europa.eu/project/id/101003914', 'https://cordis.europa.eu/project/id/101017304', 'https://cordis.europa.eu/project/id/101007326', 'https://cordis.europa.eu/project/id/101003491', 'https://cordis.europa.eu/project/id/101033819', 'https://cordis.europa.eu/project/id/101016902', 'https://cordis.europa.eu/project/id/101003826', 'https://cordis.europa.eu/project/id/955317', 'https://cordis.europa.eu/project/id/101004633', 'https://cordis.europa.eu/project/id/872427', 'https://cordis.europa.eu/project/id/101006249', 'https://cordis.europa.eu/project/id/101005122', 'https://cordis.europa.eu/project/id/952851', 'https://cordis.europa.eu/project/id/855187', 'https://cordis.europa.eu/project/id/892429', 'https://cordis.europa.eu/project/id/865067', 'https://cordis.europa.eu/project/id/883520']

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

OK that works. It's a bit complicated for me sorry. I understood the usefulness of the wait but I do not understand the usefulness of this part 'WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='o-btn o-btn--large o-btn--light co-cookie-consent__button co-cookie-consent__button--allow']"))).click()' there, nor how you find the link inside.
Let's discuss the issue in Selenium room.
0

I hope thi swill solve your question, also, please let me if this will not work for you.

elems = driver.find_elements(By.XPATH,"//a[@href]")
for elem in elems:
    print(elem.get_attribute("href"))
Improve this answer

1 Comment

Yes I tried this one but i have all the href in the page not just those who look like this one '/project/id/881603'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.