python selenium find child elements in loop

Question

I need to parse some child elements in all parent elements on page.

Create list of all articles on page

article_elements = driver.find_elements_by_tag_name('article')

And after tying to get child elements in for loop and append all results to list

for article in article_elements:
    title = article.find_element_by_xpath('//article/h2').text
    share_count = article.find_element_by_xpath('//footer/div/a/span').text
    poinst = article.find_element_by_xpath('//footer/div[2]/div[1]/div[3]').text
    meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst})

After loop ends i got 40 times one same article meta (of first article)

{'share count': u'66', 'points': u'53 points', 'title': u'25+ Random Acts Of Genius Vandalism'}
{'share count': u'66', 'points': u'53 points', 'title': u'25+ Random Acts Of Genius Vandalism'}
{'share count': u'66', 'points': u'53 points', 'title': u'25+ Random Acts Of Genius Vandalism'}
{'share count': u'66', 'points': u'53 points', 'title': u'25+ Random Acts Of Genius Vandalism'}
... 40 times

My whole code

 # coding: utf8
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time

driver = webdriver.Chrome()
driver.set_window_size(1024,768)
driver.get('http://www.boredpanda.com/')

time.sleep(2)

meta_info_list = []

article_elements = driver.find_elements_by_tag_name('article')

for article in article_elements:
    title = article.find_element_by_xpath('//article/h2').text
    share_count = article.find_element_by_xpath('//footer/div/a/span').text
    poinst = article.find_element_by_xpath('//footer/div[2]/div[1]/div[3]').text
    meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst})

for list in meta_info_list:
    print(list)

alecxe · Accepted Answer · 2016-04-06 13:27:55Z

13

The XPath expression in the loop has to start with a dot to be context-specific:

for article in article_elements:
    title = article.find_element_by_xpath('.//article/h2').text
    share_count = article.find_element_by_xpath('.//footer/div/a/span').text
    poinst = article.find_element_by_xpath('.//footer/div[2]/div[1]/div[3]').text
    meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst})

As a side note, you can shorten the code by using a list comprehension:

meta_info_list = [{
    'title': article.find_element_by_xpath('.//article/h2').text,
    'share count': article.find_element_by_xpath('.//footer/div/a/span').text,
    'points': article.find_element_by_xpath('.//footer/div[2]/div[1]/div[3]').text
} for article in article_elements]

answered Apr 6, 2016 at 13:27

alecxe

476k127 gold badges1.1k silver badges1.2k bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

python selenium find child elements in loop

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related