Let Selenium Render JS Data and Scrape?

Question

Lets say I have a web page like this:

https://store.playstation.com/en-gb/product/EP0002-CUSA24267_00-CODCWSTANDARD001

When rendered in a normal browser, this text appears:

However, when using selenium like this: https://pastebin.com/raw/MUkeMWhK

The javascript does not seem to render, it actually just shows this:

Adding a print statement just shows an object

<selenium.webdriver.remote.webelement.WebElement (session="2cfa798e8ae12131f8ad97de7f9d53af", element="92ba9953-0cd4-4f00-a560-42eb4b556348")>

How would I be able to get the text?

dimay · Accepted Answer · 2020-08-31 19:45:35Z

0

Try it:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

PATH = "C:\Program Files (x86)\chromedriver.exe"
url = "https://store.playstation.com/en-gb/product/EP0002-CUSA24267_00-CODCWSTANDARD001"

driver = webdriver.Chrome(PATH)
driver.get(url)

try:
    prodPageDealLength = WebDriverWait(driver, 15).until(
        EC.presence_of_element_located((By.CSS_SELECTOR, '.preorder-timer'))
    )
    print(driver.find_element_by_css_selector(".preorder-timer").text )
finally:
    driver.quit()

answered Aug 31, 2020 at 19:45

dimay

2,8341 gold badge17 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Arundeep Chohan · Accepted Answer · 2020-09-01 00:36:26Z

0

print(driver.find_element_by_xpath("//div[contains(@class, 'preorder-timer)]").text)

It should print the text of the preorder div class.

answered Sep 1, 2020 at 0:36

Arundeep Chohan

9,9895 gold badges17 silver badges36 bronze badges

Collectives™ on Stack Overflow

Let Selenium Render JS Data and Scrape?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related