0

Here is my code:

import multiprocessing
import dill

class Some_class():

    class_var = 'Foo'

    def __init__(self, param):
        self.name = param

    def print_name(self):

        print("we are in object "+self.name)
        print(Some_class.class_var)

def run_dill_encoded(what):
    fun, args = dill.loads(what)
    return fun(*args)


def apply_async(pool, fun, args):
    return pool.apply_async(run_dill_encoded, (dill.dumps((fun, args)),))


if __name__ == '__main__':

    list_names = [Some_class('object_1'), Some_class('object_2')]

    pool = multiprocessing.Pool(processes=4)
    results = [apply_async(pool, Some_class.print_name, args=(x,)) for x in list_names]
    output = [p.get() for p in results]
    print(output)

It returns error:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Python34\lib\multiprocessing\pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "C:\...\temp_obj_output_standard.py", line 18, in run_dill_encoded
    return fun(*args)
  File "C:/...temp_obj_output_standard.py", line 14, in print_name
    print(Some_class.class_var)
NameError: name 'Some_class' is not defined
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:/...temp_obj_output_standard.py", line 31, in <module>
    output = [p.get() for p in results]
  File "C:/...temp_obj_output_standard.py", line 31, in <listcomp>
    output = [p.get() for p in results]
  File "C:\Python34\lib\multiprocessing\pool.py", line 599, in get
    raise self._value
NameError: name 'Some_class' is not defined

Process finished with exit code 1

The code works fine without line print(Some_class.class_var). What is wrong with accessing class variables, both objects should have it and I don't think processes should conflict about it. Am I missing something? Any suggestions on how to troubleshoot it? Do not worry about run_dill_encoded and apply_async, I am using this solution until I compile multiprocess on Python 3.x.

P.S. This is already enough, but stackoverflow wants me to put more details, not really sure what to put.

12
  • 1
    Oi! I ran your code with python2.7 and python3.4 and it returned with zero: we are in object object_1 Foo we are in object object_2 Foo [None, None] Commented Apr 23, 2016 at 23:54
  • @krysopath. I am running it on python 3.4 and still getting an error Commented Apr 24, 2016 at 0:41
  • 1
    The only obvious differerence between us, that I use debian, while you are using a Microsoft OS of some version, could be a hint. Perhaps you installed some buggy libs or your python interpreter is broken. Your code runs just fine here. You could try to run it it in Cygwin first and in a virtual machine with some other non Microsoft OS after, to check if it is working there. Commented Apr 24, 2016 at 0:51
  • 1
    I believe your trouble is being caused by the fact that multiproccessing in windows has no fork like unices do. Following this accepted answer might help you: stackoverflow.com/questions/6596617/… Commented Apr 24, 2016 at 13:37
  • 2
    Hi I'm the dill and multiprocess author. If you are on non-Windows and want to check the Windows "fork" behavior, you can use dill.check (on Windows it does the same as dill.copy, of course). BTW. dill.copy is dill.loads(dill.dumps(object)). I'd suggest using multiprocess, however, as it's just more direct. Commented Apr 24, 2016 at 18:04

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.