1

Is there a way in Python to create Python code inside the Python script and then execute/test it?

My function has the following type of form (as an example)

def f(n):
    if n<=3: return [0, 0, 6, 12][n]
    return 2*f(n-1) - 4*f(n-2) - 5*f(n-3) + 15*f(n-4)

But I want to be able to create these kinds of functions dynamically (or any arbitrary function for that matter) and then test their outputs during runtime (as opposed to copying/pasting this function into the script and then manually testing it).

Not sure if this makes sense, please ask for elaboration if needed. I've already looked into eval and exec but couldn't get them to work with entire function definitions, just basic statements like 1+2, etc.

Improve this question
4
  • 2
    What you're looking to accomplish is called metaprogramming. Commented Dec 10, 2012 at 4:30
  • At which point do you want to be able to create these functions? From "user input" during runtime? Or dynamically as in you supply a plugin module that at startup will load a bunch of functions? Commented Dec 10, 2012 at 4:37
  • You need to be more specific. How are you creating these functions? Does it have to be in the form of program text rather than, e.g., a locally-defined function (or lambda), or data that drives a different function? If you really need to evaluate text on the fly, there are ways to do it (there are even ways to create a function out of hand-compiled bytecode…), but you should avoid doing so if possible, because it's usually the wrong answer in Python. Commented Dec 10, 2012 at 4:37
  • @abarnert Right now I am creating the functions in text and outputting them so I can copy/paste them into another program to test their outputs. It would save me a ton of time to just test them with the program itself. It's a recursive function that calls itself with some starter conditions (like the one in the OP) Commented Dec 10, 2012 at 4:39

3 Answers 3

Reset to default
4

There are a number of ways to do this kind of thing.

If the function can be described without "stepping outside the language", you can just define a local function and return it, as in Blender's answer. This is usually what you want when you think you need to define new functions (borrowing Blender's example):

def make_func(a, b):
    def f(n):
        return n**a + b
    return f

Sometimes, you can do even better, and represent the functions as data. For example, how do you create an arbitrary polynomial function? Well, you don't have to; you can have a generic polynomial function that takes a list of coefficients and a value and evaluates it; then all you need to do is create coefficient lists.

In fact, I think this is the one you want here. As you say:

It may be return 2*f(n-1) - 4*f(n-2) - 5*f(n-3) + 15*f(n-4) one minute, or return f(n-1) + 3*f(n-2) another, or f(n-1)+f(n-2)+f(n-3)+f(n-4)+5*f(n-5) depending on what I need it to be.

This can definitely be represented as a list of coefficients:

def make_recursive_func(coefficients, baseval):
    def f(n):
        if n < len(coefficients): return baseval[n]
        return sum(coefficient * f(n-i-1) for i, coefficient in enumerate(coefficients))
    return f

But it's probably even simpler to write a single eval_recursive_func(coefficients, baseval), if all you're ever going to do with the returned function is call it immediately and then forget it.

Sometimes—rarely, but not never—you really do need to execute code on the fly. As Himanshu says, eval and exec and friends are the way to do this. For example:

newcode = '''
def f(n):
    if n<=3: return [0, 0, 6, 12][n]
    return 2*f(n-1) - 4*f(n-2) - 5*f(n-3) + 15*f(n-4)
'''
exec(newcode)

Now the f function has been defined, exactly as if you'd just done this:

def f(n):
    if n<=3: return [0, 0, 6, 12][n]
    return 2*f(n-1) - 4*f(n-2) - 5*f(n-3) + 15*f(n-4)

It's a bit different in Py3 than Py2, and there are variations depending on what context you want things executed in, or whether you just want it executed or evaluated or compiled or treated like an import, etc. But this is the basic idea.

If you can't think of why you'd want to write the first instead of the second, then you don't need this.

And if you can't figure out how to generate the right string on the fly, you shouldn't be doing this.

And, as Ignacio Vazquez-Abrams points out, if these functions can be built out of user input, you need to do something to validate that they're safe, usually by compiling iteratively and walking the AST.

Finally, even more rarely, you need to use the new module (and/or inspect) to create a new function object on the fly out of bits of other function objects (or even out of hand-crafted bytecode). But if you need to know how to do that, you probably already know how.

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

Whoa, this works! Holy sweet mother, this worked flawlessly. Does it need the exact same spacing/format/etc? If I am creating this on the fly, do I just need \t for tabs and \r\n for newlines?
Well, you really shouldn't have tabs in Python code. And you should have \n rather than \r\n. But whatever you use, your strings should be exactly the same thing that you'd put in a script or type into the interpreter. Of course you can use \t and \n escapes instead of ''' strings. Meanwhile, I don't think this is what you actually want to do… especially if you don't want to learn about the compile, ast, etc. modules… but if I'm wrong, this is how you do it.
I think your "make_recursive_func" approach is probably superior, though. How do I actually test f(n) with it though? I tried outputting make_recursive_func(coefficients,baseval)(5) to print f(5) for example
You write func = make_recursive_func(blah blah), then call func(5). Or, if you don't need to save it, just make_recursive_func(blah blah)(5). The key here is that functions are just values—you can pass them around, store them in variables, etc. Just like you can do d[3] on a list or dict value, you can do f(3) on a function value.
I should point out that my make_recursive_func is just a sketch of the idea, not an attempt to implement your exact logic (especially with the base case, because I'm not sure what you want there). Blender's answer seems to be closer to a real implementation. And you'd use it the exact same way you'd use mine, of course.
|
3

If your functions are similar, you can create them using another function:

def make_func(a, b):
    def f(n):
        return n**a + b

    return f

Using make_func transforms this function definition:

def g(n):
     return n**2 + 1

Into just this:

g = make_func(2, 1)

In your case, something like this should work:

def create_f(start_condition, vars, coeff_pairs):
    def x(n):
        if n <= start_condition:
            return vars[n]

        result = 0.0

        for coeff, shift in coeff_pairs:
            result += coeff * x(n + shift)

        return result

    return x

And you can call it with:

f = create_f(3, [0, 0, 6, 12], [(2, -1), (-4, -2), (-5, -3), (15, -4)])

The output matches your hardcoded function's output.

Improve this answer

13 Comments

Right, but sometimes there may be different starter conditions, number of conditions, number of recursive subcomponents, etc.
@KaliMa: Can you give some examples?
It may be return 2*f(n-1) - 4*f(n-2) - 5*f(n-3) + 15*f(n-4) one minute, or return f(n-1) + 3*f(n-2) another, or f(n-1)+f(n-2)+f(n-3)+f(n-4)+5*f(n-5) depending on what I need it to be. Same sort of thing goes for the initial conditions. Could be n<=5: return [1,2,3,4,5,6][n], etc -- anything at all.
@KaliMa: You can build that pretty easily with another function.
@Blender: I think you want + shift. Or you want to pass positive shifts instead of negative ones. Otherwise, you end up with f(10) calling f(11), f(12), … forever, which explains KailMa's infinite recursion error. (Also, since all of the OP's examples seem to use -1, -2, …, -n as the shifts, I'm not sure you need to pass them explicitly in the first place… That's what I used the n-i-1 and enumerate(coefficients) for, although it probably isn't as clear as your version.)
|
2

It makes perfect sense. Python even has a set of modules explicitly for this purpose. Make sure you walk the AST and validate the nodes before executing the function though, to make sure that someone hasn't snuck in a os.system('rm -rf /') in there.

Improve this answer

2 Comments

I actually came across this page via Google but couldn't find anything that would let me synthesize actual functions -- just expressions like 3+4, etc.
@KaliMa: Assuming you're in Python 2, the eval function can only evaluate expressions, not statements (or suites), and you can't define a function in an expression (except a lambda—which doesn't work here, because, among other things, you need it to be recursive).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.