Wxpython multi-threading is running threads sequentially not all at once

Question

import wx
import os
import sys
import wx.lib.plot as plot
import datetime
import urllib
import threading

pathstr = '/media/meant2b/My Passport/C Drive/Convert/GFSMaps/'

class Frame(wx.Frame):
    def __init__(self, parent, id, title):
        style = wx.DEFAULT_FRAME_STYLE ^ (wx.RESIZE_BORDER)
        wx.Frame.__init__(self, parent, id, title=title, size = (1024, 768), style=style)
        self.Center()
        self.panel = wx.Panel(self)
        self.panel.Bind(wx.EVT_KEY_DOWN, self.OnKeyDown)
        self.panel.SetFocus()
        self.panel.SetBackgroundColour('White')
        self.bitmap = None
        self.PhotoMaxSize_1 = (881)        
        self.CreateMenuBar()
        self.model = 0
        self.map = 0
        self.dMinus = 0
        self.Model = ['00', '06', '12', '18']
        self.MDay = ['000','003','006','009','012','015','018','021','024','027','030','033','036','039','042','045','048','051','054','057','060','063','066','069','072','075','078','081','084','087','090','093','096','099','102','105','108','111','114','117','120','123','126','129','132','135','138','141','144','147','150','153','156','159','162','165','168','171','174','177','180','183','186','189','192','204','216','228','240','252','264','276','288','300','312','324','336','348','360','372','384']
        self.MapType = 0


    def gif1(self, event):
        pathstr = '/'
        for self.dMinus in range(4, -1, -1):
            self.MoveCalendar() #sets date
            for Counter1 in range(0, 4):
                for Counter2 in range(0, 81):
                    if os.path.isfile(pathstr + self.strDate + self.Model[Counter1] + self.MDay[Counter2] + '.gif') == True:
                    continue
                    url = 'http://mag.ncep.noaa.gov/GemPakTier/MagGemPakImages/gfs/' + self.strDate + '/' + self.Model[Counter1] + '/gfs_namer_' + self.MDay[Counter2] + '_1000_500_thick.gif'
                    urllib.urlretrieve(url,str(pathstr + self.strDate + self.Model[Counter1] + self.MDay[Counter2] + '.gif'))
                    statinfo = os.stat(str(pathstr + self.strDate + self.Model[Counter1] + self.MDay[Counter2] + '.gif'))
                    if statinfo.st_size<21000L:
                        os.remove(str(pathstr + self.strDate + self.Model[Counter1] + self.MDay[Counter2] + '.gif'))
                        counter2 = 80

         for Counter1 in range (3, -1, -1):
             url = 'http://mag.ncep.noaa.gov/GemPakTier/MagGemPakImages/gfs/' + self.Model[Counter1] + '/gfs_namer_384_850_temp_mslp_precip.gif'
             urllib.urlretrieve(url,pathstr + '850Temp384.gif')
             statinfo = os.stat(pathstr + '850Temp384.gif')
             if statinfo.st_size<21000:
                 os.remove(pathstr + '850Temp384.gif')
                 continue
             else:
                for Counter2 in range(0, 81):
                    url = 'http://mag.ncep.noaa.gov/GemPakTier/MagGemPakImages/gfs/' + self.Model[Counter1] + '/gfs_namer_' + self.MDay[Counter2] + '_850_temp_mslp_precip.gif'
                    urllib.urlretrieve(url,pathstr + '850Temp' + self.MDay[Counter2] + '.gif')
                    statinfo = os.stat(pathstr + '850Temp' + self.MDay[Counter2] + '.gif')
                    if statinfo.st_size<21000:
                        os.remove(pathstr + '850Temp' + self.MDay[Counter2] + '.gif')
        print('SHOW ME')
        self.VGFS(event)

    def gif2(self, event):
        url = 'http://www.cpc.ncep.noaa.gov/products/precip/CWlink/daily_ao_index/ao.sprd2.gif'
        urllib.urlretrieve(url,pathstr + '/AO.gif')

        url = 'http://www.cpc.ncep.noaa.gov/products/precip/CWlink/pna/nao.sprd2.gif'
        urllib.urlretrieve(url,pathstr + '/NAO.gif')

        url = 'http://www.cpc.ncep.noaa.gov/products/precip/CWlink/pna/pna.sprd2.gif'
        urllib.urlretrieve(url,pathstr + '/PNA.gif')

        url = 'http://www.esrl.noaa.gov/psd/enso/mei/ts.gif'
        urllib.urlretrieve(url,pathstr + '/MEI.gif')
     '''
     And numerous more gifs/pngs as well
     '''

    def All(self, event):
        dl0 = threading.Thread(target = self.GFS(event))
        print('MESHOW')
        dl0.start()

        dl1 = threading.Thread(target = self.Oscillators(event))
        print('MEGO')
        dl1.start()

class App(wx.App):

    def OnInit(self):
        self.frame = Frame(parent = None, id =-1)
        self.frame.Show()
        self.SetTopWindow(self.frame)
        return True

if __name__=='__main__':
    app = App()
    app.MainLoop()

I'm skipping tons of code. When I run the above in wxpython it runs fine EXCEPT, it runs the code sequentially not all at the same time. Each thread is downloading numerous gif & png files...enough that I want to split up the action to save some time. It's making the call to each 'thread' but it's waiting for dl0 to finish before it runs dl1. I can tell by looking at the directory where the files are being stored. How do I change this around to get it so it will call both threads 'at the same time?'

I will add this in as well as I don't know if this might be something different in threading in wxpython vs. tkinter. In tkinter to get this to work I never had to add in Thread.init(self)(this won't let me do the __ for some reason) like I see being done in wxpython coding I've seen.

Pretty much I'm just trying to take two separate running correctly programs and combine them together into one right now. I am changing from tkinter to wxpython as well.

We'd have to see what gifs1 and gifs2 are doing to diagnose. — FogleBird
– FogleBird, Commented Feb 14, 2014 at 21:35

Community · Accepted Answer · 2020-06-20 09:12:55Z

In the line

dl0 = threading.Thread(target = self.GFS(event))

you are assigning the result of the computation self.GFS(event) to target. While it is theoretically possible that this result is a callable object, it is unlikely, and I guess what you really mean is instead

dl0 = threading.Thread(target=self.GFS, args=[event])

The same applies to dl1, of course.

Some additional remarks on threading

Suppose I have some simple function with some side effects and the return value None, like this:

>>> import threading
>>> def foo():
        print "I am FOO!"

Now I can start this function in a separate thread, like so:

>>> t = threading.Thread(target=foo)
>>> t.start()
I am FOO!

As soon as I start the thread t with its start method, it looks if target is defined and callable, and since that's the case, it calls it, and foo prints its text.

Now what happens if I erroneously assign not foo itself, but foo() to the parameter target? Let's see:

>>> t = threading.Thread(target=foo())
I am FOO!
>>> t.start()

The very moment I perform the assignment, Python has to compute the value of foo(), which happens to be None (since foo has no return value), and during this computation, the side effects of foo are triggered.

But when I start the threads t with start(), then target has the value None, which isn't callable, and so nothing at all happens.

In your case, I suspect that whatever magic self.GFS(event) does, it is done within the main thread during the assignment, while dl0.start() does absolutely nothing at all. Which explains why the magic of GFS and Oscillators appears to be happening sequentially: they both happen within the main thread, while the two helper threads do nothing at all.

Some additional remarks on wx

It is certainly possible to mix wx and threading, but the mixture has some rough edges. For example, it might be desirable for unittest purposes to create and destroy several wx.App objects in separate threads, but wxWidgets (and therefore wxPython) is strongly built on the assumption that there is only one wx.App-Object ever, and that it resides in the main thread.

I prefer to separate heavy computations (which should get their own thread, of course) and all wx-Gui-stuff as much as possible. You might want to consider a pattern like the following:

First, the GUI-side:

# ...

import wx.lib.newevent

ComputationDoneEvent, EVT_COMPUTATION_DONE = wx.lib.newevent.NewEvent()

# ...

class MainFrame(wx.Frame):

    def __init__(self):
        # ...
        self.Bind(EVT_COMPUTATION_DONE, self.OnComputationDone)

    def OnMainButtonPressed(self, evt):
        # The user pressed the big fat button and wants
        # some heavy computation performed
        t = threading.Thread(target=heavy_computation,
                             args=[self, arg1, arg2],
                             daemon=True)
        t.start()
        self.statusbar.SetStatusText("Busy doing stuff...")
        # eventually disable input, change cursor to busy,
        # whatever is appropriate in your case, and how
        # much it makes sense for the user to proceed
        # with other stuff while waiting for the result.

    def OnComputationDone(self, evt):
        # this gets called later when the computation is done
        self.statusbat.SetStatusText("Result: " + str(evt.result))
        # enable input, change cursor, whatever

Now, the heavy computation side:

def heavy_computation(mother, arg1, arg2):
    # First, all the stuff that has nothing to do with
    # the GUI
    perform_some_side_effects()
    result = some_more_computations(arg1, arg2)

    # Now we have to inform the wx side that we are done
    evt = ComputationDoneEvent(result=result)
    wx.PostEvent(mother, evt)

Of course for really complex and tedious stuff, you will want to provide some kind of progress meter, and perhaps you want to display exceptions raised within heavy_computation somewhere on your GUI. But I hope you get the idea. See also http://wxpython.org/Phoenix/docs/html/lib.newevent.html for a more complete example.

Mike Driscoll · Accepted Answer · 2014-02-14 21:26:01Z

0

I can't really tell why that doesn't work without seeing more of the code. I did create a wxPython downloading application last month though that you can take a look at for inspiration though:

http://www.blog.pythonlibrary.org/2014/01/29/wxpython-creating-a-file-downloading-app/

Basically I just subclass Thread from the threading module, have it download a file using requests and then update the wxPython GUI. To start a download, I call my download thread with the url and a couple of other parameters. I can do that multiple times with no problems.

One of my readers took the example and extended it a bit here:

https://github.com/Yoriz/WxPython-Downloader

Hope that helps.

answered Feb 14, 2014 at 21:26

Mike Driscoll

33.2k9 gold badges51 silver badges91 bronze badges

2 Comments

confused Over a year ago

I'm starting to wander though...I keep reading and seeing the same thing...running ONE thread, not multiple threads. I'm trying to run multiple threads at the same time. Right now I'm not trying to do anything with the program other than downloading the images while I go off and do other things, which is the intent of the program anyways. I want to be able to run multiple threads while I'm off surfing the web/doing other things in complete separate programs. Once the program gets finished then I'll go about looking at what got downloaded. I haven't seen anyone using multiple threads yet.

Mike Driscoll Over a year ago

I added url after url to download multiple files at once and it seemed to work. But if you want true parallel downloading, then you probably want to use multiprocessing.

Collectives™ on Stack Overflow

Wxpython multi-threading is running threads sequentially not all at once

2 Answers 2

Some additional remarks on threading

Some additional remarks on wx

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Some additional remarks on threading

Some additional remarks on wx

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related