Pass object method as lambda or partial function, replacing self parameter with target object?

Question

Use case: Lets say that I have a very large set of Pandas Series objects and I wish to apply the .drop_na() method to all of them. However because they are so large I must use a function that will perform this with multiprocessing, by accepting a list of objects and using each object in the list as the argument to a method (which is also passed as an object).

Given that the first, and implicit, argument to an object method is self.

Can I use a lambda, partial or other trickery to pass the .drop_na() method and replace the self argument in .drop_na() with a specific instance of a series? Is this possible in Python?

Mad Physicist · Accepted Answer · 2017-04-06 21:12:59Z

2

Yes. The first argument is only "implicit" when the method is bound to an object that is an instance of the class using . notation, e.g.

sr = Series(...)
sr.drop_na()

In this case, sr.drop_na is actually a closure that replaces the self parameter of Series.drop_na with a reference to sr, sort of like

lambda: Series.drop_na(sr)

You can always call the method from the class directly and pass the self parameter explicitly. The example above can be rewritten as:

sr = Series(...)
Series.drop_na(sr)

You can pass the function Series.drop_na around as necessary and call it on whatever objects you need. Notice that Series.drop_na takes a single parameter, while sr.drop_na does not.

This is not trickery and is in fact a deliberate design choice in the language that is intended to be used for cases just like yours (and many others as well).

edited Apr 6, 2017 at 21:12

answered Apr 6, 2017 at 19:26

Mad Physicist

116k29 gold badges202 silver badges292 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Jeremy Barnes Over a year ago

Thank you. I feel like closures in python are not discussed enough.

Mad Physicist Over a year ago

Yes, the rules can be a bit obscure at times. This particular one is pretty simple, but it took me a long time to realize that a different closure is created every time you access a method via . on the object: stackoverflow.com/q/41900639/2988730

chepner Over a year ago

Nitpick: sr.drop_na is the closure; sr.drop_na() is the application of the closure to its remaining required arguments (of which there are none).

Mad Physicist Over a year ago

@chepner. Good catch, fixed.

Collectives™ on Stack Overflow

Pass object method as lambda or partial function, replacing self parameter with target object?

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related