Web Scraping tradingview chart with python selenium

Question

I am trying to scrape data from tradingviews charts. This is the code that I am using and so far is working

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import time

MAMAarray = []
FAMAarray = []

# example option: add 'incognito' command line arg to options
option = webdriver.ChromeOptions()
option.add_argument("--incognito")

# create new instance of chrome in incognito mode
browser = webdriver.Chrome(chrome_options=option)

# go to website of interest
browser.get("https://www.tradingview.com/chart/vKzVQllW/#")

# wait up to 10 seconds for page to load
timeout = 10
try:
    WebDriverWait(browser, timeout).until(EC.visibility_of_element_located((By.XPATH, "/html/body/div[1]")))
except TimeoutException:
    print("Timed out waiting for page to load")
    browser.quit()


time.sleep(5)

# get MAMA and FAMA values for BTCUSDT on binance
MAMA_element = browser.find_element_by_xpath("/html/body/div[1]/div[1]/div/div[1]/div/table/tbody/tr[1]/td[2]/div/div[3]/div[3]/div/span[3]")
FAMA_element = browser.find_element_by_xpath("/html/body/div[1]/div[1]/div/div[1]/div/table/tbody/tr[1]/td[2]/div/div[3]/div[3]/div/span[4]")

MAMAarray.append(float(MAMA_element.text))
FAMAarray.append(float(FAMA_element.text))

print(MAMAarray)
print(FAMAarray)

while True:
    MAMA_element = browser.find_element_by_xpath(
        "/html/body/div[1]/div[1]/div/div[1]/div/table/tbody/tr[1]/td[2]/div/div[3]/div[3]/div/span[3]")
    FAMA_element = browser.find_element_by_xpath(
        "/html/body/div[1]/div[1]/div/div[1]/div/table/tbody/tr[1]/td[2]/div/div[3]/div[3]/div/span[4]")
    if (float(MAMA_element.text)) != MAMAarray[-1]:
        MAMAarray.append(float(MAMA_element.text))
    if (float(FAMA_element.text)) != FAMAarray[-1]:
        FAMAarray.append(float(FAMA_element.text))

    print(MAMAarray)
    print(FAMAarray)

Here is the output: so you can see that the numbers append but ONLY when I go into the chart manually and move my cursor over new candles(bars)

[7415.969]
[7417.39]
[7415.969, 7428.644]
[7417.39, 7435.585]
[7415.969, 7428.644, 7430.56]
[7417.39, 7435.585, 7431.722]
[7415.969, 7428.644, 7430.56, 7415.496]
[7417.39, 7435.585, 7431.722, 7417.39]
[7415.969, 7428.644, 7430.56, 7415.496, 7415.969]
[7417.39, 7435.585, 7431.722, 7417.39]
[7415.969, 7428.644, 7430.56, 7415.496, 7415.969]
[7417.39, 7435.585, 7431.722, 7417.39]
[7415.969, 7428.644, 7430.56, 7415.496, 7415.969]
[7417.39, 7435.585, 7431.722, 7417.39]
[7415.969, 7428.644, 7430.56, 7415.496, 7415.969]
[7417.39, 7435.585, 7431.722, 7417.39, 7424.887]
[7415.969, 7428.644, 7430.56, 7415.496, 7415.969, 7439.161]
[7417.39, 7435.585, 7431.722, 7417.39, 7424.887, 7424.409]

How do I go about automating this so that it automatically gets the newest value instead of me having to cursor over the new bar for it to get the new value

EDIT:

This is the value Im looking for Like I said I can get it to output through the script, but I cant get the value to update when a new candle is made

this

Is the problem resolved , I could not make it to work.Can you share the solution and working code completely here. — Marx Babu
– Marx Babu, Commented Aug 1, 2019 at 10:26

Andrei · Accepted Answer · 2018-06-02 07:58:00Z

4

If I correct understood you, you need a hoverover an element. To be able to do it, you can try this:

hover = ActionChains(webDriver).move_to_element(element_to_hover_over)
hover.perform()

Also you can have a look to Documentation

EDIT:

In your case I would use this two selectors to get the information you need:

//span[@class='dl-ask']
//span[@class='dl-bid']

They are changing dynamically without hover need.

Output will be like this:

7420.06x2 // you have to split this string and extract 7420.06
7424.00×0.007977999746799469 // and here 7424.00

Since you want to get dynamically from elements you provided, there is a solution for it. I wouldn't use it for every problem, because it is not a "clean" one. But for our specific case I think it is a best one.

I have found that if you hover to particular candle in the graphic, every minute when graphic will be updated, all candles will be shifted to the left. Also if you hover on (x,y) position which is (y) position for one of the candles, you get the value of the candle. And that is exactly what you need. My propose is to hover to the candle you need and the values from your elements will be updated since the values this candle be updated(every 1 minute). If you want the information with minimum delay, you have to hover on the right candle.

Use this to move to the mouse to your candle:

move_to_element_with_offset(to_element, xoffset, yoffset)

Move the mouse by an offset of the specified element. Offsets are relative to the top-left corner of the element. Args:

to_element: The WebElement to move to.

xoffset: X offset to move to.

yoffset: Y offset to move to.

Also you may want to zoom in the graphic. You can try the suggestions here, or:

webdriver.execute_script("window.scrollBy(0, 150);") // 0 - xPos, 150 - yPos (values are in px).

edited Jun 2, 2018 at 7:58

answered Jun 1, 2018 at 15:19

Andrei

5,6675 gold badges27 silver badges51 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Logan Thompson Over a year ago

seems like what I need to do but I am having a hard time figuring out what element changes when each new candle comes out. How do I figure this out?

Andrei Over a year ago

@LoganThompson I have added some information. Please have a look

Logan Thompson Over a year ago

This is the information I am trying to extract right here which I have found the correct xpath for, problem is the script loads the web browser and stays on the current candle. Like you were saying the hover method might do it for me just unsure where or how to get the script to hover from one candle to the next as they are added on a 1 minute basis. I have added another image to show what I am looking at.

Andrei Over a year ago

@LoganThompson the solution to your problem I have posted, please have a look on updated unswer

Andrei Over a year ago

@LoganThompson did it help you?

|

Collectives™ on Stack Overflow

Web Scraping tradingview chart with python selenium

1 Answer 1

8 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related