How to stop multithreading returning combined data using python

Question

When using a multithreading, I get combined data
The list is : A,B,C. If I MT this, the fdata [] contains data from A,B and C. How do I Get fdata too hold only one set of data. I tried del fdata didnt help. I need some kind of lock.

class WorkerThread(threading.Thread):
       def __init__(self, queue):
            threading.Thread.__init__(self)
            self.queue = queue

       def run(self):
            while 1:
              try: # take a job from the queue
                   symbol, test, test2 = self.queue.get_nowait()

              except Queue.Empty:
                   raise SystemExit

              fn = %s.CSV" % symbol
              fdata = []
              fo = open(fn, 'rb')
              fr = csv.reader(fo, dialect='excel')
              for row in fr:
                   fdata.append(row)
              #print fdata 
              #del fdata

How would I add the thread number to fdata or list id A,B,C to fdata?

What exactly is multi threaded here? Is your fdata global in some way? — Liorsion
– Liorsion, Commented Feb 8, 2011 at 6:59

Petriborg · Accepted Answer · 2011-02-09 13:24:20Z

1

fdata should always contain the contents of the CSV file, after all you did for-loop over the rows, so it should always include A B and C... Maybe you should explain more of what you're trying to do.

As to your second question - Your thread object has an ident see thread.get_ident()

Return the ‘thread identifier’ of the current thread. This is a nonzero integer. Its value has no direct meaning; it is intended as a magic cookie to be used e.g. to index a dictionary of thread-specific data. Thread identifiers may be recycled when a thread exits and another thread is created.

Edit:

Maybe fdata is somehow a global variable or being saved across accesses to the file? I see nothing in the code snippet that would do that, but I'm at a loss to otherwise explain it. According to the function fdata should be a locally scoped variable in the function, and it should just go away with the stack frame...

edited Feb 9, 2011 at 13:24

answered Feb 8, 2011 at 15:14

Petriborg

2,9704 gold badges29 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Merlin Over a year ago

Trying do to: simple have a lot of csv's. all are pretty much the same. symbol.csv is file name. "symbol" changes. I want to multithread: the file read then load into db. I need fdata to contain onlt data from one file. As it stands now, fdata being open and used by all threads its being loaded all subquent csv data.

Collectives™ on Stack Overflow

How to stop multithreading returning combined data using python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related