Getting span text(value) every time it changes in selenium python

Question

I am trying to print the value of a span every time it changes. To print the value of the span is quite easy:

popup = wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="spot"]')))

Print(popup.text)

This will print the value at that moment, the problem is that the value will change every 2 seconds. I tried using:

# wait for the first popup to appear
popup = wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="spot"]')))
# print the text
print(popup.text)
# wait for the first popup to disappear
wait.until(EC.staleness_of(popup))

# wait for the second popup to appear
popup = wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="spot"]')))
# print the text
print(popup.text)
# wait for the second popup to disappear
wait.until(EC.staleness_of(popup))

No matter how long my wait value is, 10 or 20 or even 30 seconds, the process always times out. I do not know much about coding but I think this method does not work because the span as a whole does not change only the span value(text). One method that I tried was to loop the Print(popup) command and it partially worked. it printed the same value 489 times until it changed and printed the other one 489 times again.I have since tried this code:

popup = wait.until(EC.text_to_be_present_in_element_value((By.XPATH, '//*[@id="spot"]')))
print(popup.text)

but it returns:

TypeError: __init__() missing 1 required positional argument: 'text_'

.

Please help what it is I need to add or what method I need to use to get the changing value.

HTML code inspection

Please I beg you, please beware Im not trying to print the text of the span, I already know how to do that, I want print it everytime it changes

Lennart Thamm · Accepted Answer · 2020-09-18 12:10:30Z

1

Assuming that the element does disappear and reappear again:

You can just go back and forth between waiting for the element being located and being located.

Assuming that the elements content changes, but doesn't disappear:

I don't know of any explicit way to wait for the change of the content of an element, so as far as I am concerned you would need to compare the change yourself. You might want to add an absolute wait of < 2 seconds to limit the amount of unnecessary comparisons you make.

# Init a list to contain the values later on
values = []

# Wait for the element to be loaded in the first place
popup = wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="spot"]')))

values.append(popup.text)


while True:

    # possibly wait here
    
    new_value = driver.find_element(By.XPATH, '//*[@id="spot"]')
    
    # look up if the value has changed based on the values you know and add the new value
    if values[-1] != new_value:
        values.append(new_value)

    # add an exit condition unless you actually want to do it forever

Please be aware: This will only work if the value actually changes each and every time or if you don't need duplicates that follow one another. If you need every value, you can leave out the comparison and add one value every ca. 2 seconds.

For your example: The page on binary.com you provided uses websocket in order to refresh the content. This is a protocol that allows the server to send data to the client and the other way around. So it's a different approach to the http protocol you are used to (you send a request, the server replies - let's say you ask for the webpage, then the server will just send it).

This protocol opens a connection and keeps it alive. There will hardly be a wait to anticipated this change. But: In your browser (assuming Chrome here) you can go into your developer tools, go into the "Network" Tab and filter for the WS (websocket). You'll see a connection with v3?app_id=1 (you might need to refresh the page to have output in the Network-Tab). Click on that connection and you'll see the messages your client sent annd the ones you received. Naturally you only need those received so filter for those.

As those are quite a few steps have a look on that screenshots, it shows the correct settings:

Every message is in json format and you click on it to see its content. Under "tick" you'll see the ask and bid data. In case that suffices, you can just leave the page open for as long as you need, then copy the output, save it as a file and read it with python for analysis.

It seems you can also automate this with selenium as demostrated here: http://www.amitrawat.tech/post/capturing-websocket-messages-using-selenium/

Basically they do the same thing, they set the capability to record the log, then filter through it to get the data they need. Note that they use Java to do so - but it wont be hard to translate to python.

edited Sep 18, 2020 at 12:10

answered Sep 17, 2020 at 9:52

Lennart Thamm

2891 silver badge14 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Moshe Over a year ago

Thank for your answer, I cannot use the comparison method because there are duplicates. Also I have used the method of adding the value every 2 seconds, it works but my problem with it is accuracy and timing, suppose the script reads and prints a number exactly at (sec: milliseconds) 00:999 and there are other commands between the first print and the next print, the next print wont be at (seconds:milliseconds)02:999 as it supposed to, thus printing incorrectly. The reason I needed the staleness method was because its accurate to print every 02:000 when the number(element content) has changed

Moshe Over a year ago

I have not use system time, I have always used time.sleep(2) and loop

Lennart Thamm Over a year ago

Can you share the page? It would be good to see what happens when new content is being loaded.

Moshe Over a year ago

please inspect the small chart

Collectives™ on Stack Overflow

Getting span text(value) every time it changes in selenium python

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related