How to continuously stream data points from dynamically growing list using matplotlib.animation for real time graphing?

Question

I am wanting to plot a line from my data in real-time using matplotlib.animation module for python. The overview of my code is that I am calculating a score a storing it in a list("my_average") and this is the y-coordinate. The score will always be between -1 & +1. further more I would like my x-coordinate to be the length of my list "(len(my_average))".Ex; list recieves one score x-coordinate would the length of the list so 1 and y-coordinate would be the score, list receives second score, list plots (1, score1) (2, score2), etc. I am not able to show the graph and need help with this portion of my code. Further more if possible I would not like to read the list from a csv file but rather directly from memory and still be able to view previous data points in history.

Here is the code:

from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
import json
from textblob import TextBlob
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import time
import numpy

# Variables that contains the user credentials to access Twitter API
access_token = 
access_token_secret = 
consumer_key = 
consumer_secret = 


# This is a basic listener that just prints received tweets to stdout.
my_list = [] #creates empty list
my_average = []
class StdOutListener(StreamListener):

    def on_data(self, data):
        json_load = json.loads(data)
        texts = json_load['text'] # string
        #print(texts)
        wiki = TextBlob(texts)
        r = wiki.sentiment.polarity
        my_list.append(r)
        #drop zero in list
        if 0 in my_list: my_list.remove(0)  
        print (my_list)   

        #calculate average
        average = numpy.mean(my_list)
        b = my_average.append(average)
        #drop "nan" from list
        if 'nan' in my_average: my_average.remove('nan')
        print "average", my_average

        fig = plt.figure()
        ax = plt.axes(xlim=(0, 10), ylim=(0, -10))
        line, = ax.plot([], [], lw=2)

        def init():
            line.set_data([], [])
            return line,

# animation function.  This is called sequentially
        def animate(i):
            x = (len(my_average))
            y = (my_average)
            line.set_data(x, y)
            return line,

            anim = animation.FuncAnimation(fig, animate, init_func=init,frames=100, interval=20, blit=True)
            plt.show() 

            return True

        def on_error(self, status):
            print(status)

auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
stream = Stream(auth, StdOutListener())

# This line filter Twitter Streams to capture data by the keywords: 'python', 'javascript', 'ruby'
stream.filter(track=['USD/CAD', 'Dollar', 'Loonie' ], languages=['en'])

Thank you in advance.

Julia Schwarz · Accepted Answer · 2015-07-06 15:52:09Z

1

I don't have a twitter key to test your full code, however here is a code sample that you can adapt to animate a plot. A disadvantage of this approach is that you need to kill the process to stop the animation. This example plots the average along with the raw data, for illustration.

import numpy
from pylab import *
import time

class StdOutListener():
  def __init__(self):
    self.start_time = time.time()
    self.x = []
    self.y = []
    self.my_average = []
    self.line_actual, = plot(self.x, self.y)                  # line stores a Line2D we can update
    self.line_average, = plot(self.x, self.my_average)       # line stores a Line2D we can update


  def on_data(self, new_value):
    time_delta = time.time() - self.start_time                # on our x axis we store time since start
    self.x.append(time_delta)
    self.y.append(new_value)
    self.my_average.append(numpy.mean(self.y))
    self.line_actual.set_data(self.x, self.y)
    self.line_average.set_data(self.x, self.my_average)
    ylim([min(self.y), max(self.y)])        # update axes to fit the data
    xlim([min(self.x), max(self.x)])
    draw()                                  # redraw the plot

ion()                                       # ion() allows matplotlib to update animations.

out_listener = StdOutListener()
for i in range(1000):
  out_listener.on_data(i + numpy.random.randint(-5,5))

edited Jul 6, 2015 at 15:52

answered Jul 5, 2015 at 17:39

Julia Schwarz

2,6502 gold badges20 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

RustyShackleford Over a year ago

Thank you for your response. I have implemented your code and I keep running into error "line 75, in <module> out_listener.on_data(i + random.randint(-5,5)) AttributeError: 'builtin_function_or_method' object has no attribute 'randint'" I do not understand this error can you please advise? Thank you very much

Julia Schwarz Over a year ago

this looks like a namespace collision. Your 'random' module may not be the default python 'random' module. Try using numpy.random.randint(-5,5) instead. I've updated the code.

RustyShackleford Over a year ago

Wow! Hell ya it works beautifully! Thank you so much saving my butt!

RustyShackleford Over a year ago

If possible could you please explain what and why we have added this line in your code and what it does: for i in range(1000): out_listener.on_data(i + numpy.random.randint(-5,5) Thank you in advance.

Julia Schwarz Over a year ago

Sure, i'm just simulating new data coming in, but with noise. If it were just on_data(i), you'd see a straight line. This way the line is jagged, closer to data you actually observe.

|

Collectives™ on Stack Overflow

How to continuously stream data points from dynamically growing list using matplotlib.animation for real time graphing?

1 Answer 1

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related