Scrape data from a highchart using selenium

Question

I am trying to scrape data from a website which is on a highchart,I have tried to use class names and it always prints an empty list

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pandas as pd

driver = webdriver.Chrome('/Users/MR NAT ASS/Downloads/chromedriver')
driver.get("https://www.statsbots.org.bw/motor-vehicle-accident-trend-transport-sector")
data = driver.find_elements_by_xpath('//g[@class="html not-front not-logged-in no-sidebars page-node 
page-node- page-node-105 node-type-easychart mean-container"]')

year_stats = []
for p in range(len(data)):
year_stats.append(data[p].text)

years = driver.find_elements_by_xpath('//g[@class="highcharts-axis-labels highcharts-xaxis-labels"]')

year_list = []
for s in range(len(years)):
year_list.append(years[s].text)

print(year_stats)
#####
print(year_list)

Any suggestions please

here is the website statsbots.org.bw/motor-vehicle-accident-trend-transport-sector — ditiro
– ditiro, Commented Jun 1, 2021 at 12:00
Check if you can get the JSON values in the page and look for window.easychart = it contains the data used for generate the chart. Also, browse cb.botswana.opendataforafrica.org/nanhvtg/… and check if the data is available through requests. — Marco Aurelio Fernandez Reyes
– Marco Aurelio Fernandez Reyes, Commented Jun 1, 2021 at 21:25

ppotaczek · Accepted Answer · 2021-06-02 07:46:00Z

1

All charts are stored in global variable Highcharts. You can get series data for a specific chart by running the below JS code:

Highcharts.charts[2].userOptions.series[0].data;

answered Jun 2, 2021 at 7:46

ppotaczek

39.3k2 gold badges20 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Scrape data from a highchart using selenium

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related