python scrape ajax content

Question

I'd like to parse website with rates but i couldn't take out data from <td> elements.

I wrote short code to test which gets 1st line with data tabel:

from selenium import webdriver

driver = webdriver.Firefox()
driver.get('https://www.gpw.pl/wskazniki_spolek_full')
table = driver.find_elements_by_xpath("//table[@class='tab03']/tbody/tr")[4].text
print table

driver.quit()

and i'm getting results:

2 PLNFI0800016 141 08OCTAVA 42 786 848 44,07 63,86 2016-12-31 H 0,69 27,80 ---

but i'd like go through all <td> elements in <tr> tag in loop by all tables which has class='tab03'

table = driver.find_elements_by_xpath("//table[@class='tab03']/tbody/tr")

for el in table:
    col_id =  el.find_element_by_tag_name('td')[1].text
    col_kod = el.find_element_by_tag_name('td')[2].text

    print("{}".format(col_id, col_kod))

driver.quit()

but i'm getting error: selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: td

alecxe · Accepted Answer · 2017-04-03 13:59:29Z

1

There are some header rows that don't have td elements inside them, skip them:

rows = driver.find_elements_by_xpath("//table[@class='tab03']/tbody/tr")

for row in rows[3:]:
    cells = row.find_elements_by_tag_name('td')
    col_id = cells[0].text
    col_kod = cells[1].text

    print("{}".format(col_id, col_kod))

Also note that, to get to the td cells, use the find_elements_by_tag_name() and get the desired elements by index (0-based).

answered Apr 3, 2017 at 13:59

alecxe

476k127 gold badges1.1k silver badges1.2k bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

python scrape ajax content

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related