0

I am trying to get the next page button from this site "https://remoters.io/?s=finance..I have tried switching the iframe which I think is the right iframe..still no luck. I have tried finding this button with find_element_by_xpath etc.

next_iframe = driver.switch_to_frame('aswift_2')
next = next_iframe.find_element_by_partial_link_text('Next')

Can anyone have a look and help. Thanks.

2 Answers 2

1

When I go to "https://remoters.io/?s=finance" I do not see an iframe. I can get to the second page with the code below. time.sleep(5) can be removed if you write some extra code to make sure the modal is gone before trying to click next.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Chrome(executable_path=r'C:\\Path\\To\\chromedriver.exe')
driver.get("https://remoters.io/?s=finance")

agree = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//button[contains(text(),"AGREE")]')))
agree.click()

next = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//a[contains(text(),"Next")]')))
while "page/2/" not in driver.current_url:
    next.click()
Sign up to request clarification or add additional context in comments.

3 Comments

I was kind of hoping to avoid that..Is it necessary to use Wait?..I think the button loads with the page..It's not something..that loads after..can you have a look?
@Abhishek Rai "Wait" makes the scrip more stable incase the website loads slow which happens eventually. Updated the answer to handle the issue I saw where the popup modal fading from view would cause the script to finish without getting to the second page.
Unfortunately My PC got the blue screen of death. No way for me to check...:(..
0

using the code:

driver = webdriver.Firefox()
driver.get('https://remoters.io/?s=finance')
driver.find_element_by_class_name('next').click()

I can click to the next page.

Explanation

In order to find out how to select a specific element I use ScrapeMate Beta on Firefox, which gives me the higher level classes directly. Another option is to use inspect element:

enter image description here

Now we can see the following:

  1. It is a link, but there are multiple 'a' elements.
  2. It is stored under the 'page-numbers' class. But this one is also not specific enough (can use find_elements and then filter over the text input)
  3. The class name next, which is a unique class on this page, so I picked that value.

Try to keep full path searches to the bare minimum, they are relatively slow.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.