0

I started this question off in a previous post: Web scraping with Selenium not capturing full text

If you look at the bottom of that page, I need to write a while loop that scrolls to the bottom of each page and grabs the full text.

This is what I have so far:

while True:

for i in tqdm(chat_links):
     driver.get(i)
     driver.execute_script("window.scrollTo(0,document.body.scrollHeight)")
    try:
         elements = driver.find_element_by_class_name('text').text
         temp={'elements':elements}
         chat_text.append(temp)
    except:
     driver.close()
Improve this question
5
Related questions
1
Can't make a loop run in my script
0
Webscraping with selenium
1
For Loops while using selenium for webscraping Python
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.