0

What I am trying to do is get all the followers of an instagram user. The problem is, the followers show up in a dialog box that is dynamically loading once you get to the bottom. I found solutions to get to the bottom of an infinite window like in: window scrolling or get to te bottom of a dialog box like in: Pop up scrolling

But none of those work in my case.

This is my code:

import os
import random
import sys

from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys

url = 'https://www.instagram.com/inbaral/followers/'
driver = webdriver.Firefox()
driver.implicitly_wait(1)
driver.get(url)


username = driver.find_element_by_xpath('//*[@name="username"]')
password = driver.find_element_by_xpath('//*[@name="password"]')
login_btn = driver.find_element_by_xpath('//*[@class="_0mzm- sqdOP  L3NKy       "]')

username.send_keys("name")
password.send_keys("pass")

#login
login_btn.click()
WebDriverWait(driver, 20).until(EC.url_changes('https://www.instagram.com/accounts/login/?next=/inbaral/followers/'))

#click on followers
followers_btn= driver.find_elements_by_class_name('g47SY')
followers_btn[1].click()

# extract followers
WebDriverWait(driver, 20).until(EC.url_changes('https://www.instagram.com/accounts/login/?next=/inbaral/followers/'))
elem = driver.find_element_by_xpath('//*[@class="FPmhX notranslate _0imsa "]')
elem.send_keys(Keys.END)

soup = BeautifulSoup(driver.page_source,features="lxml")
followers = soup.find_all("a",{'class':'FPmhX notranslate _0imsa '})

for follower in followers:
    print(follower.get_text())

driver.quit()

Note: Another problem is that sometimes the loading doesn't work when getting to the bottom, and the only way to fix it is by physically playing with the scrollbar

1
  • So how do you handle ...physically playing with the scrollbar... while Test Execution takes place? Commented Dec 29, 2018 at 20:38

1 Answer 1

0
dialog = driver.find_element_by_xpath("/html/body/div[4]/div/div[2]/div/div")

This never worked:

dialog.send_keys(Keys.END)

This worked for me(if I'm logged in):

driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", dialog)

If i don't log in none of them works

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.