2

This below code gets data from a table in a web page.

After crawling through a page it goes to next page and does the same thing again. The URL of the page doesn't change while moving to next page.

I want to use a loop so that it goes on for 50 or 75 times and breaks.

    driver.get(site)
    mytable = driver.find_element_by_css_selector('.table.table...nline')
    for row in mytable.find_elements_by_css_selector('tr'):
        for cell in row.find_elements_by_tag_name('td'):
            sys.stdout=open("abcd.txt","a+")
            print(cell.text)
            sys.stdout.close()
     driver.find_element_by_xpath("//li[@class='button next']/a").click()

I have tried using while loop, but i'm getting issues while appending the file.

1 Answer 1

1

Try slicing:

driver.get(site)
mytable = driver.find_element_by_css_selector('.table.table...nline')
for row in mytable.find_elements_by_css_selector('tr')[:50]:
    for cell in row.find_elements_by_tag_name('td'):
        sys.stdout=open("abcd.txt","a+")
        print(cell.text)
        sys.stdout.close()
driver.find_element_by_xpath("//li[@class='button next']/a").click()
Sign up to request clarification or add additional context in comments.

1 Comment

@thoris i added a [:50] on line 3

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.