2

Hello i have 3 functions f1(), f2() and f3(). The output of the previous is the input of the next. meaning output = f3(f2(f1(data))).

instead of writing

def outp(data):
    o1 = f1(data)
    o2= f2(o1)
    o3 = f3(02)
    return o3
output=outp(data)

is there a way to this by simply providing a list of functions to some other general function and let it handle the chaining together?

7 Answers 7

3

You could simply run a for loop with an assignment:

>>> f1 = int
>>> f2 = float
>>> f3 = lambda x: x * 2
>>> i = '3'
>>> for func in (f1, f2, f3):
...     i = func(i)
...     print(i)
... 
3
3.0
6.0
Sign up to request clarification or add additional context in comments.

Comments

3

I stumbled across your question and I found it really interesting.

Here is my approach on a pipeline of functions:

def call_pipeline(transition_list, *args):
    """
    transition_list = [func1, func2, func3...
                       ]
    - func1 output is the input for func2 etc
    """

    result = args
    for f in transition_list:
        result = f(*result)

    print(*result)

def f1(x, y):
    return [x + y]

def f2(z):
    return [z**2, z]

def f3(w, r):
    return [w * r]

def f4(t):
    return ["Final", t]

def test_pipeline():
    transition_list = [f1, f2, f3, f4]
    call_pipeline(transition_list, *[1, 2])

As long as you make sure that every function inside the pipeline returns a list and the next function can properly process the output of the previous one, this should work fine.

Comments

2

It is fairly easy to define a compose function that handles simple single-argument functions:

def compose(*args):
    if not args:
        return lambda x: x  # Identity function
    else:
        return compose(args[0], compose(args[1:]))

outp = compose(f3, f2, f1)  # Note the order of the arguments

You can also use the reduce function (functools.reduce in Python 3):

outp = reduce(lambda f, g: lambda x: f(g(x)), [f3, f2, f1], lambda x:x)

You can omit the third argument if you are certain the list of functions won't be empty.

Comments

2

You can define a composition function that operates on a sequence of functions:

def compose(a, b):
    def fcn(*args, **kwargs):
        return a(b(*args, **kwargs))
    return fcn

def compose_all(*fcns):
    return reduce(compose, fcns)

def compose_all_1(*fcns):
    return reduce(compose, fcns, lambda x: x)

The function compose is the basic building block that takes two functions and returns their composition. With this elementary idiom you can extend this to an arbitrary sequence of functions in compose_all. The variant compose_all_1 works even on a 1- or 0-element input sequence.

Comments

1

I wrote a library called Pipeline Functions to solve this exact problem. Here's what it would look like for the simple example in the question:

from pipeline_func import f
output = data | f(f1) | f(f2) | f(f3)

Functions that take more than one argument are also supported. Here's an example that includes functions like zip(), which takes two arguments, and map()/filter(), which expect the input from the pipeline as the second argument rather than the first:

>>> from pipeline_func import f, X
>>> a = [3, 4, 2, 1, 0]
>>> b = [2, 1, 4, 0, 3]
>>> a | f(zip, b) | f(map, min, X) | f(filter, bool, X) | f(sorted)
[1, 2, 2]

Comments

0

You can define a pipeline builder, that you can reuse to build pipelines.

Each pipeline is a function that can be invoked with one argument or none (rarely used). When a pipeline is called it sequentially run every functions with previous function return value as argument.


from functools import reduce
import warnings

# pipeline function builder
def pipe(*funcs):
    """
Usage:
pipe(
  lambda x: x+1,
  lambda x: x+20
)(50)

Order of execution of functions is FIFO.
FIFO = First In First Out
    """
    # if no arguments return an identity function and warn the developer
    if not funcs:
        warnings.warn("""pipe() is useless when called without arguments.
Please provide functions as arguments or remove call to pipe().
        """)
        return lambda x=None: x # Identity function
    
    def runner(value=None):
        return reduce(lambda acc, curr: curr(acc), funcs, value)
    return runner


# test 1
pipeline = pipe()
print(pipeline())
# None


# test 2
pipeline = pipe()
print(pipeline(8))
# 8


# test 3
pipeline = pipe(
    lambda unused: 50
)
print(pipeline())
# 50


# test 4 -> Real Usage
pipeline = pipe(
    lambda age: age >= 18,
    lambda is_adult: 'Adult' if is_adult else 'Minor'
)
print(pipeline(12))
# Minor
print(pipeline(20))
# Adult

Comments

0

Since you want to define functions in a list, functool.reduce is exactly what you are looking for:

from functools import reduce
pipeline = [f1, f2, f3]
out = reduce(lambda val, fun: fun(val), pipeline, data)

reduce takes 3 input arguments respectively:

function: Here is defined by lambda

Sequence: a sequence of values the function should execute on. Here, this sequence of values is a sequence of functions.

initial: The initial value for executing the function (first input argument or lambda function) in the first run. (=data)

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.