0

I have a seemingly simple Python code design problem that I have not found any elegant solution to. I have a set of modules (two in the example below) that define functions of the same name that compute the same thing but using different algorithms:

algorithm_a.py:

def compute_1(x):
   # do some computation
   return (x+1.)**2

def compute_2(x):
   # do some other computation

# etc

algorithm_b.py:

def compute_1(x):
   # do same computation in another way
   return x*x + 2.0*x + 1.0

def compute_2(x):
   # do some other computation in another way

# etc

Both modules have approximately 10 functions, but the number might grow in the future.

In a third module, I have functions defined that are not aware of the details of how the computation is performed, i.e. they only care about getting the correct answer. The interface to the user is provided by a class. All in all, the third module looks similar to:

simulation.py:

import XXX as algo #????

def do_computation_1(x):

   v1 = algo.compute_1(x)
   v2 = algo.compute_2(x)

   # Do some computations using v1, v2 
   return result

def do_computation_2(x):
   return algo.compute_2(x)

# etc etc

class Simulation(object):

  def __init__(self, method):
     # Do some magic here so that
     # if method == "A":
     #   algo = algorithm_a
     # if method == "B"
     #   algo = algorithm_b

  def run(self, x):
     do_computation_1(x)
     do_computation_2(x)

How can I get the correct module loaded and applied in do_computation() to depend on the method parameter supplied to the class?

Note that the do_computation functions need to remain outside of the Simulation class.

CONCLUSION: The comment by @BrenBarn below summarises well the available options. Thanks all for the great help!

Improve this question
11
  • You could just import the modules inside of the __init__. So, if method == 'A': import algorithm_a as algorithm. Commented Sep 18, 2015 at 16:50
  • You might be very interested in this post, which is a near duplicate Commented Sep 18, 2015 at 16:51
  • @TobiaTesan Thanks, good idea! The Strategy pattern would indeed be applicable here, I was looking for perhaps something more straight-forward Commented Sep 18, 2015 at 17:47
  • @MorganThrapp But algorithm would not be visible outside the object, i.e. to the do_computation() function, right? Commented Sep 18, 2015 at 18:19
  • Why do you need to numba jit the do_computation function instead of the compute_x functions in each algo module? Commented Sep 18, 2015 at 18:57

2 Answers 2

Reset to default
1

The better way to do this is to actually save (or pass) the function you want to use.. E.g.

import algorithm_a
import algorithm_b


class Simulation(object):

  def __init__(self, method):
     # Do some magic here so that
     if method == "A":
       self.compute_func = algorithm_a.compute
     if method == "B"
       self.compute_func = algorithm_b.compute

  def run(self, x):
     self.compute_func(x)

If you really must have your external def do_computation(x) function you can pass the algorithm you want to use as an argument

def do_computation(x, compute):
   return compute(x)

class Simulation(object):

  def __init__(self, method):
     # Do some magic here so that
     if method == "A":
       self.compute_func = algorithm_a.compute
     if method == "B"
       self.compute_func = algorithm_b.compute

  def run(self, x):
     do_computation(x, self.compute_func)
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks for the suggestion! Unfortunately, in my use case there are a bunch of functions in algorithm_a and algorithm_b which are called by several functions similar to do_computation(). So this becomes quickly cumbersome
Then save a reference to the module rather than the function and pass the module to do_computation
I'm afraid that won't work either since the do_computation function is actually decorated with Numba @jit, passing algo to it would probably cause Numba not to be able to compile it in nopython mode (haven't tried it though..)
0

With the way your code is structured, you can't have the import depend on a parameter passed to the class. Python files are executed top-to-bottom. Your import occurs before the class definition, so by the time the class is defined, the import has already occurred. (And you won't be passing method in until you instantiate the class, which will be even later.)

If it's okay to import both modules and you just want to use the specified one, you can do almost literally what you wrote in the comments:

import algorithm_a
import algorithm_b

class Simulation(object):

    def __init__(self, method):
        if method == "A":
            self.algo = algorithm_a
        if method == "B"
            self.algo = algorithm_b

    def do_computation(self, x):
        return self.algo.compute(x)

    def run(self, x):
        self.do_computation(x)

(I have here made do_computation a method of the class. It doesn't make much sense to have it as a separate function if its behavior is determined by a parameter passed to the class.)

If the actual import of the modules may be slow, you could conditionally import one module or the ohter as shown in Reblochon's answer. However, to do this you must put the imports inside the class. If you are going to specify the algorithm via something like Simulation("A"), there's no way to determine which import to do at the time you do import simulation, because you haven't yet specified the algorithm at that time; you would have to wait until you actually call Simulation("A"). (If the import is slow, this will cause a slowdown at that point, which may not be desirable.)

Edit: if you really need to have do_computation be a global function because of Numba, you can work around it by setting a global variable. Change the __init__ to:

def __init__(self, method):
    global algo
    if method == "A":
        algo = algorithm_a
    if method == "B"
        algo = algorithm_b

And make a global do_computation function like this:

def do_computation(x):
    return algo.compute(x)

This will be more confusing because every time you create a new Simulation it will change the global behavior, even for previously-created simulations. But if you aren't creating multiple simulations with different algorithms at the same time, it should be okay.

Improve this answer

5 Comments

Thanks for the detailed description of why my structuring of the code is making what I want difficult! As you say, do_computation() should be a method of a class, and that is the way I would want it. Unfortunately, that function is jitted using Numba, and currently this is not supported for member functions (except for older versions of Numba which I can't use for other reasons). So currently I have to keep it as a separate function.
@user435548: I made an edit to show how you can set a global variable instead of an attribute to work around that.
Interesting idea using a global. Unfortunately I will indeed need to create several simulations with different algorithms. It starts to look like I will have to make a bigger change to my code..
@user435548: Then you'll probably have to pass the algorithm module (or its compute function) as an argument, as in Chad Simmons's answer. But from your comments it seems like there are a number of requirements that you didn't state in your original question (e.g., you actually have more than one compute function in each algo module). You should maybe edit your question to be more explicit about what your actual requirements are. A solution that works okay for importing two modules with one function each may not scale well if you have lots of modules with lots of functions.
I edited the question to better indicate that I in fact have several compute function in the two modules.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.