Python: Parallelizing a nested for loop

Question

I'm a little lost in trying to use the multiprocessing module. I have a simple nested loop to do some copying of attributes from one object to another:

        if objects:
            rb_from = obj_act.rigid_body
            # copy settings
            for o in objects:
                rb_to = o.rigid_body
                if o == obj_act:
                    continue
                for attr in self._attrs:
                    setattr(rb_to, attr, getattr(rb_from, attr))

If nothing else, I'd like to parallelize the inner loop, but it's not clear to me how to do that. Most of the examples here focus on using the map function of multiprocessing, but I don't really care about the return value from setattr, I just want those calls to execute in parallel.

You can parallelize function call, but not for loops. You will have to reorganize your code a bit. Also your payload seems to be rather small, with the overhead of spawning processes it might even get slower. — Klaus D.
– Klaus D., Commented Feb 9, 2021 at 3:37
And additionally all processes have their own variables. Setting an attribute will not be reflected in the main process. — Klaus D.
– Klaus D., Commented Feb 9, 2021 at 3:45
This is a terrible use of multithreading. All of your threads depend on a single object. You are best off parallelizing the outer loop, and I don't think you're going to gain very much there, either. — Frank Yellin
– Frank Yellin, Commented Feb 9, 2021 at 6:38

ghchoi · Accepted Answer · 2021-02-09 04:51:33Z

1

You may not achieve what you want using multiprocessing. (Or, it could be rather complex.) Variables out of the context are not shared among processes. But they are among threads.

You can simply use threading and parallelize copying executions.

    def copy_attributes(self, obj_act, o): 
        rb_from = obj_act.rigid_body

        rb_to = o.rigid_body
        if o == obj_act:
            return
        for attr in self._attrs:
            setattr(rb_to, attr, getattr(rb_from, attr))

    ...

            for o in objects:
                threading.Thread(
                    target=self.copy_attributes, args=(obj_act, o)
                ).start()

answered Feb 9, 2021 at 4:51

ghchoi

5,2064 gold badges38 silver badges59 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python: Parallelizing a nested for loop

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related