PYTHON : Finding hidden element in html source code

Question

I'm trying get the url below which is hidden inside the page source code at https://www.aliexpress.com/item/32212764152.html but it's hidden inside a script tag.

<script>
                    window.runParams = {"descriptionModule":{"descriptionUrl":"https://aeproductsourcesite.alicdn.com/product/description/pc/v2/en_US/desc.htm?productId=32212764152&key=HTB1GwO_aVY7gK0jSZKzM7OikpXac.zip&token=f32528ddd34e37aecddda1c7778d5f0c"} .... </script>

I've manage to get the source code but not sure how to extract the url as an object.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
import re

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
CHROMEDRIVER_PATH = '/Users/reezalaq/PycharmProjects/wholesale/driver/chromedriver'
options = Options()
options.headless = False
driver = webdriver.Chrome(CHROMEDRIVER_PATH, options=options)
driver.get('https://www.aliexpress.com/item/32212764152.html')

html = driver.page_source

def run_script():
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    body = driver.find_element_by_css_selector('body')
    body.send_keys(Keys.PAGE_UP)

count = 0
while count < 3: #13
     run_script()
     count+=1
     time.sleep(5)

x = html.startswith('https://aeproductsourcesite.alicdn.com')
print(x)

How do I filter everything else in the source code and have an object ?

x = "https://aeproductsourcesite.alicdn.com/product/description/pc/v2/en_US/desc.htm?productId=32212764152&key=HTB1GwO_aVY7gK0jSZKzM7OikpXac.zip&token=f32528ddd34e37aecddda1c7778d5f0c"

Sers · Accepted Answer · 2019-10-23 08:24:34Z

2

You can use Regular Expression to extract the value:

import re
#..
url = re.compile(r'"descriptionUrl":"([^"]*)"').search(html).group(1)

answered Oct 23, 2019 at 8:24

Sers

12.3k2 gold badges14 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

PYTHON : Finding hidden element in html source code

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related