Having trouble in getting data using css selector/xpath in selenium

Question

I'm trying to extract data from the link below using selenium via python: www.oanda.com

But I'm getting an error that, "Unable to Locate an Element". In browser console i tried using this Css selector:

document.querySelector('div.position.short-position.style-scope.position-ratios-app')

This querySelector returns me the data for short percentage of 1st row in the browser console(for this test), but when i used this selector in the python script below it gives me an error that, "Unable to Locate element" or sometimes empty sctring. Please suggest me solution if there's any.Will be grateful, thanks :)

# All Imports
import time
from selenium import webdriver

#will return driver
def getDriver():
    driver = webdriver.Chrome()
    time.sleep(3)
    return driver


def getshortPercentages(driver):
    shortPercentages = []
    shortList = driver.find_elements_by_css_selector('div.position.short-position.style-scope.position-ratios-app')
    for elem in shortList:
        shortPercentages.append(elem.text)
    return shortPercentages

def getData(url):
    driver = getDriver()
    driver.get(url)
    time.sleep(5)
    # pagesource = driver.page_source
    # print("Page Source: ", pagesource)
    shortList = getshortPercentages(driver)
    print("Returned source from selector: ", shortList)


if __name__ == '__main__':
    url = "https://www.oanda.com/forex-trading/analysis/open-position-ratios"
    getData(url)

Andersson · Accepted Answer · 2018-10-16 10:04:00Z

1

Required data is located inside an iframe, so you need to switch to iframe before handling elements:

driver.switch_to.frame(driver.find_element_by_class_name('position-ratios-iframe'))

Also note that data inside iframe is dynamic, so make sure that you're using Implicit/Explicit wait (using time.sleep(5) IMHO is not the best solution)

edited Oct 16, 2018 at 10:04

answered Oct 16, 2018 at 9:58

Andersson

52.8k18 gold badges83 silver badges132 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

GigaByte Over a year ago

thanks it's working. was working on that from recent 4hours and you save my day. Many thanks.

Collectives™ on Stack Overflow

Having trouble in getting data using css selector/xpath in selenium

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related