Python - How to print URL using .click/selenium

Question

My code goes into a webpage, and can click on each row of the page.

I want to ultimately, open each link of each row as a new page, so I can save the URL of each page.

I cannot get the URL without Seleniums .click() (i believe), so I resorted to .click, and then get the currentURL.

This code works only for the first row, as the webpage exits the original URL.

Is there a more efficient way to print the URLS of each row?

from selenium import webdriver
import time
sam=[]
import pandas as pd
driver = webdriver.Chrome()
for x in range (1,8):
    driver.get(f'https://www.abstractsonline.com/pp8/#!/9325/presentations/endometrial/{x}')
    time.sleep(3)
    page_source = driver.page_source
    eachrow=driver.find_elements_by_xpath('//*[@id="results"]/li')
    for item in eachrow:
        oth = item.find_element_by_xpath(".//h1[@class='name']")
        oth.click()
        print(driver.current_url)

Are you trying to click on an <h1> tag? If you have an <a> tag, you can fetch the href attribute from it without clicking on it. — Tim Roberts
– Tim Roberts, Commented Mar 18, 2021 at 6:17

Arundeep Chohan · Accepted Answer · 2021-03-18 06:30:30Z

1

<h1 class="name" data-id="696" data-key="">
  <span class="title color-primary">
    <i class="icon-caret-right"></i> 
    <span class="bodyTitle">Panel Discussion</span>
  </span>
</h1>

Each of your elements has a data-id grab all of them and just append url

https://www.abstractsonline.com/pp8/#!/9325/presentation/696

https://www.abstractsonline.com/pp8/#!/9325/presentation/{}

You can then grab them by

eachrow=["https://www.abstractsonline.com/pp8/#!/9325/presentation/"+x.get_attribute('data-id') for x in driver.find_elements_by_xpath('//*[@id="results"]/li//h1[@class='name']')]

for row in eachrow:
    driver.get(row)

answered Mar 18, 2021 at 6:30

Arundeep Chohan

9,9895 gold badges17 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python - How to print URL using .click/selenium

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related