Get all the attributes "title" with selenium in python

Question

Until now I used a for cycle to get all the elements on a page in a certain path with this script:

for username in range(range_for_like):

    link_username_like = "//article/div[2]/div[2]/ul/div/li[" + str(num) + "]/div/div[1]/div/div[1]/a[contains(@class, 'FPmhX notranslate zsYNt ')]"
    user = browser.find_element_by_xpath(link_username_like).get_attribute("title")
    num += 1
    sleep(0.3)

But sometimes my cpu will exceed 100%, which is not ideal.

My solution was to find all the elements in one line using find_elements_by_xpath but in doing so, I can't figure out how to get all the "title" attributes.

I know that the path changes for every title, //article/div[2]/div[2]/ul/div/li[" + str(num) + "]/div/div[1]/div/div[1]/a that's why I kept increasing the num variable, but how can I use this tecnique without a cycle for?

What's the most efficient way in term of performance to get all the attributes? I don't mind if it does take also 2 minutes or more

driver.fiind_elements_by_xpath(//@title); This will return all the elements on page with title attribute, store in list and iterate over it to get its value using get_attribute method... — Amit Jain
– Amit Jain, Commented Sep 9, 2018 at 13:36
@sers the page I that i'm trying to parse is an instagram page, I want to obtain all the people that put a like on my post, so I open my last post, and click over the like, thee i have a list of people that left a like, the name of the people are in the attribute title — alessandro buffoli
– alessandro buffoli, Commented Sep 10, 2018 at 7:06
@AmitJain I tried with user = browser.find_elements_by_xpath('//@title').get_attribute("title") but it doesn't work — alessandro buffoli
– alessandro buffoli, Commented Sep 10, 2018 at 7:07

Sers · Accepted Answer · 2018-09-10 09:22:40Z

4

Here how you can get all the people that like your photo by xpath:

//div[text()='Likes']/..//a[@title]

Code below get first 12 liker:

likes = browser.find_elements_by_xpath("//div[text()='Likes']/..//a[@title]")
for like in likes:
    user = like.get_attribute("title")

To get all likes you have to scroll, for that you can get total likes you have and then scroll until all likes will be loaded. To get total likes you can use //a[contains(.,'likes')]/span xpath and convert it to integer.

To scroll use javascript .scrollIntoView() to last like, final code would look like:

totalLikes = int(browser.find_element_by_xpath("//a[contains(.,'likes')]/span").text)
browser.find_element_by_xpath("//a[contains(.,'likes')]/span").click()

while true:
   likes=browser.find_elements_by_xpath("//div[text()='Likes']/..//a[@title]")
   likesLen = len(likes)
   if (likesLen == totalLikes - 1)
       break
   browser.execute_script("arguments[0].scrollIntoView()", likes.get(likesLen-1))

for like in likes:
    user = like.get_attribute("title")

How it works: With //div[text()='Likes'] I found unique div with window contains likes. Then to get all likes that is li I go to parent div with /.. selector and then get all a with title attribute. Because all likes not loading immediately you have to scroll down. For that I get total likes amount before click to likes. Than I scroll to last like (a[@title]) to force instagram to load some data until total likes I got not equals to list of likes. When scroll completes I just iterate throw all likes in list I got inside while loop and get titles.

edited Sep 10, 2018 at 9:22

answered Sep 10, 2018 at 8:38

Sers

12.3k2 gold badges14 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

alessandro buffoli Over a year ago

This works perfectly, but let me understand how it wors because I want to do the same thing for the comments. with the //div[text()='Likes'] you search for the main div that contains all the likes? and then you get the attribute @title in the a tag?

Sers Over a year ago

@alessandrobuffoli I tried to explain more details in my answer, hope you'll understand my poor english. You also can go with more faster solution using Javascript but more complex.

alessandro buffoli Over a year ago

Thankyou for your explanation, it has been really helpful, thankyou!

Collectives™ on Stack Overflow

Get all the attributes "title" with selenium in python

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related