Python Selenium: Unable to fetch table content

Question

I am trying to access this URL, here I have to fetch table under Price / Tax History section. Below is my code:

from selenium import webdriver
from selenium.webdriver.common.by import By

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
import os, sys
from multiprocessing import Pool
from selenium.webdriver import DesiredCapabilities
from selenium.webdriver.support.ui import WebDriverWait

driver = webdriver.Firefox()
wait = WebDriverWait(driver, 5)
driver.maximize_window()
driver.get('https://www.zillow.com/homedetails/2114-Bigelow-Ave-N-Seattle-WA-98109/48749425_zpid/')
sleep(10)
p_history = driver.find_elements_by_css_selector('#tax-price-history  table tr > td')
    for p in p_history:
        print(p.text)

it is not printing text.

Update Screen of the section required:

Update#2

Ran against PhantomJS and here you can see loader image in the section(Scroll the image)

Can you try with the following query selector, note that it's not extensible and would work on only first row, for that you would need to do few more changes: document.querySelector('#tax-price-history tbody tr td:nth-child(3)') — Anupam Saini
– Anupam Saini, Commented Mar 20, 2017 at 10:48
@AnupamSaini First off, I am using Python, second I did try this and did not work — Volatil3
– Volatil3, Commented Mar 20, 2017 at 10:50
which text you are trying to fetch exactly, there are few td which doesn't have any text, they have span and some more span inside them and then text. — Gaurang Shah
– Gaurang Shah, Commented Mar 20, 2017 at 10:51

jymbob · Accepted Answer · 2017-03-20 13:48:09Z

2

You need to tell selenium to use WebDriverWait and expected_conditions to find the element once it's loaded.

You need a reference to an element that doesn't exist on page load, but should be there once the Ajax request is complete. It looks like #tax-price-history table should fulfil that requirement.

try:

from selenium.webdriver.support import expected_conditions as EC
parent = wait.until(EC.presence_of_element_located((
    By.CSS_SELECTOR, '#tax-price-history table')))

p_history = parent.find_element_by_css_selector('td')

If the element isn't found in the time limit specified in wait you'll get an error

edited Mar 20, 2017 at 13:48

answered Mar 20, 2017 at 11:52

jymbob

4886 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Volatil3 Over a year ago

` value = method(self._driver)TypeError: 'list' object is not callable`

jymbob Over a year ago

ah. the syntax is quite different in python. I'll edit the answer

Collectives™ on Stack Overflow

Python Selenium: Unable to fetch table content

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related