getting a python function of functionType for a nested function with recursion

Question

I am trying to do unit testing for nested functions (function inside a function), I am using code("nested" is function name) from below link which would supply closures and returns a valid function that is callable from tests. It works for simple functions. I am trying to make it work for recursive functions.

As an example: I am trying to get a valid function for "innerfunction" which has an entry in co_freevars as "innerfunction".

I want to get a function (FunctionType I believe) for "innerfunction" as a callable. To get this I need to call FunctionType with a tuple with a FunctionType for "innerfunction". This becomes a recursive "dependency". How can I resolve this dependency for parameters to be sent for "closures"

Function for

def toplevelfunction():
    def innerfunction(a):
        print ('val of a is ', a)
        if a > 0:
            innerfunction(a -1)
    innerfunction(10)

Original code that I am using:

def freeVar(val):
  def nested():
    return val
  return nested.__closure__[0]

codeAttribute = '__code__' if sys.version_info[0] == 3 else 'func_code'

def nested(outer, innerName, **freeVars):
  if isinstance(outer, (types.FunctionType, types.MethodType)):
    outer = outer.__getattribute__(codeAttribute)
  for const in outer.co_consts:
    if isinstance(const, types.CodeType) and const.co_name == innerName:
      return types.FunctionType(const, globals(), None, None, tuple(
          freeVar(freeVars[name]) for name in const.co_freevars))

https://code.activestate.com/recipes/580716-unit-testing-nested-functions/

How to add support for closures so below works:

func = nested(toplevelfunction, 'innerfunction')

func(5)

would return error need a closure of length 1. Adding a closure referring to "const" shows up that it is of CodeType and not FunctionType. Adding a closure value to refer itself seems tricky after reading through the documentation.

I do find innerfunction as:

 {code} <code object innerfunction at 0x104b9bc00, file "/<filedirectory>/handle_results.py", line 44>
 co_argcount = {int} 1
 co_cellvars = {tuple} <class 'tuple'>: ()
 co_code = {bytes} b"t\x00\x00d\x01\x00|\x00\x00\x83\x02\x00\x01|\x00\x00d\x02\x00k\x04\x00r'\x00\x88\x00\x00|\x00\x00d\x03\x00\x18\x83\x01\x00\x01d\x00\x00S"
 co_consts = {tuple} <class 'tuple'>: (None, 'val of a is ', 0, 1)
 co_filename = {str} '/<fildirectory>/handle_results.py'
 co_firstlineno = {int} 44
 co_flags = {int} 19
 co_freevars = {tuple} <class 'tuple'>: ('innerfunction ',)
 co_kwonlyargcount = {int} 0
 co_lnotab = {bytes} b'\x00\x01\r\x01\x0c\x01'
 co_name = {str} 'innerfunction '
 co_names = {tuple} <class 'tuple'>: ('print',)
 co_nlocals = {int} 1
 co_stacksize = {int} 3
 co_varnames = {tuple} <class 'tuple'>: ('a',)

At the very least, you are missing the return statement that causes toplevelfunction to return the inner function, rather than returning None and discarding the function it creates. — chepner
– chepner, Commented Jan 23, 2017 at 20:22
can you elaborate? I edited above to specify how I am using "nested" from tests. — MyDevMallik
– MyDevMallik, Commented Jan 23, 2017 at 20:38
toplevelfunction doesn't do anything useful. You don't have an explicit return statement, so it implicitly returns None. No references to innerfunction exist after toplevelfunction exits. (Even if you did add return innerfunction to toplevelfunction, it's not a very interesting closure, because there are no other local variables in toplevelfunction for it to close over.) — chepner
– chepner, Commented Jan 23, 2017 at 20:43
Edited toplevelfunction. My function is a lot complicated and tried to simplify the fucntion. I can see that there are co_freevars with the function name itself. Trying to see how I can get out of this situation. — MyDevMallik
– MyDevMallik, Commented Jan 23, 2017 at 20:51
innerfunction in your edit still is not a closure, because toplevelfunction does not return it. It's just a function defined and used in the scope of top levelfunction. — chepner
– chepner, Commented Jan 23, 2017 at 21:02

Alfe · Accepted Answer · 2017-03-16 13:51:53Z

Thank you for hinting me on this limitation on my ActiveState recipe you linked to in your question!

You tried to unit test a local recursive function; my minimal example for this looks like this:

def f(x):
  def fac(n):
    return fac(n-1) * n if n > 1 else 1
  print "Faculty of", x, "is", fac(x)

So in a unit test you want to test the inner function fac(). Simply applying my recipe leads to this error:

nestedFac = nested(f, 'fac')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 7, in nested
  File "<stdin>", line 7, in <genexpr>
KeyError: 'fac'

This is because fac is an identifier used within the inner function, and you must specify the value of each identifier used within the inner function. Callable identifiers are no exception:

nestedFac = nested(f, 'fac', fac=None)

This sets fac to the value None and the call to nested does not fail anymore.

Unfortunately, the resulting function will try to call None(...) when you call it:

nestedFac(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in fac
TypeError: 'NoneType' object is not callable

You would need to pass the function itself as the value for fac to avoid this. Unfortunately, you do not have it yet when you call nested(). A repeated wrapping could be done like this:

nestedFac = nested(f, 'fac', fac=None)
nestedFac = nested(f, 'fac', fac=nestedFac)
nestedFac = nested(f, 'fac', fac=nestedFac)
nestedFac = nested(f, 'fac', fac=nestedFac)

This would limit your recursion depth to the number of wraps you applied:

nestedFac(2)
2
nestedFac(3)
6
nestedFac(4)
24
nestedFac(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in fac
  File "<stdin>", line 3, in fac
  File "<stdin>", line 3, in fac
  File "<stdin>", line 3, in fac
TypeError: 'NoneType' object is not callable

To avoid this, you can pass a local function as proxy:

def q(*args, **kwargs):
  return nestedFac(*args, **kwargs)
nestedFac = nested(f, 'fac', fac=q)

Or as a lambda:

nestedFac = nested(f, 'fac', fac=lambda *args, **kwargs:
    nestedFac(*args, **kwargs))

Or for the special case of just one parameter:

nestedFac = nested(f, 'fac', fac=lambda n: nestedFac(n))

An extension to the nested() recipe which does this automatically would be the better approach, though. Feel free to fork the recipe and add this aspect! :-)

Collectives™ on Stack Overflow

getting a python function of functionType for a nested function with recursion

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related