Global variable not accessed inside pool(multiprocessing) in python

Question

The global variable defined in main is accessed inside normal function but its not accessed in Pool()

from multiprocessing import Pool    
import functools


def inc(x):
    print( x + 1)
    
    print(headers) # GETTING ERROR HERE

def dec(x):
    print (x - 1)
    

def add(x, y):
    print (x + y)
    
    
    
def a(f):
    f()
    


def main():
    print(headers)
    
    f_inc = functools.partial(inc, 4)
    f_dec = functools.partial(dec, 2)
    f_add = functools.partial(add, 3, 4)

    
    with Pool() as pool:
        res = pool.map(a, [f_inc, f_dec, f_add])

    print(res)


if __name__ == '__main__':
    global headers
    headers = {'Accept': 'application/json'}
    main()

Expected output is

{'Accept': 'application/json'}
5
{'Accept': 'application/json'}
1
7
None

But the output i get is

{'Accept': 'application/json'}
    5

NameError: name 'headers' is not defined

The global variable is not getting accessed inside pool in multiprocessing.

akaButters · Accepted Answer · 2021-06-16 19:14:42Z

1

You can't access it in separate pool processes because each process has started its own Python interpreter, and the global variable isn't in it.

You need to pass the variables you need to each pool process. There are a number of ways to communicate between Processes. One of them is a multiprocessing.Queue(), and there are other ways too. There are many examples in the Python.org docs that show how to do this here.

edited Jun 16, 2021 at 19:14

answered Jun 16, 2021 at 19:13

akaButters

4592 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Madan Raj Over a year ago

Could you please share the code for the above example.. And link of some examples..

akaButters Over a year ago

I edited my answer with a link to the multiprocessing docs. There are at least two Pool() examples there. They show you how to use a Queue() to pass information.

Collectives™ on Stack Overflow

Global variable not accessed inside pool(multiprocessing) in python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related