How to scrape specific information from javascript webpage by Selenium Python?

Question

I can't scrape the 'Resolution' field from the javascript webpage, as I believe. Webpage address: https://support.na.sage.com/selfservice/viewdocument.do?noCount=true&externalId=60390&sliceId=1&noCount=true&isLoadPublishedVer=&docType=kc&docTypeID=DT_Article&stateId=4183&cmd=displayKC&dialogID=197243&ViewedDocsListHelper=com.kanisa.apps.common.BaseViewedDocsListHelperImpl&openedFromSearchResults=true I need to extract Description, Cause, and Resolution.

Tried various ways to get element, including:

find_element_by_xpath
find_element_by_id
find_element_by_class_name.

Nothing gave the desired result. Could you direct me in which way should I work?

Welcome to SO. Could you show us your code and what you've tried so far? In addition, it helps to include the error traceback. — trotta
– trotta, Commented Jul 18, 2019 at 9:08
elements = WebDriverWait(browser, 10).until(EC.presence_of_all_elements_located((By.XPATH, ".//*"))) for element in elements: print(element.text) — Ozzborn
– Ozzborn, Commented Jul 19, 2019 at 3:59
results_list = browser.find_elements_by_xpath("//ol[@id='extiframe']/li[@class='extiframe']") print(len(results_list)) results_list = browser.find_elements(By.CSS_SELECTOR, '#extiframe > li') print(len(results_list)) — Ozzborn
– Ozzborn, Commented Jul 19, 2019 at 4:00

Thinh Phung · Accepted Answer · 2019-07-18 09:24:59Z

1

https://support.na.sage.com/selfservice/viewContent.do?externalId=60390&sliceId=1 This is the correct url that you can crawl html, use Network tab of your browser devtool to find that. Example with Chrome

answered Jul 18, 2019 at 9:24

Thinh Phung

996 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ozzborn Over a year ago

Thank you a lot! That looks promising. Unfortunately, this website has stopped me to scrape the info. The search button is inactive. Probably, there is a rule to charge a requester after a certain number of questions, I guess.

Collectives™ on Stack Overflow

How to scrape specific information from javascript webpage by Selenium Python?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related