Pass a function/method to a python script

Question

I have defined this method/function in a google colab cell

[5]:def lstm_model2(input_features, timesteps, regularizers=regularizers, batch_size=10):

       model = Sequential()
       model.add(LSTM(15, batch_input_shape=(batch_size, timesteps, 
          input_features), stateful=True,
                   recurrent_regularizer=regularizers))
       model.add(Dense(1))
       model.compile(loss='mae', optimizer='adam')

       return  model

I want to pass this method to a script I am executing in the next cell using argparse.

[6]:!python statefulv2.py --model=lstm_model2

I tried an approach similar to type argument in argparse like defining a identically named abstract method inside the statefulv2.py script so that argparse.add_argument('--model', help='LSTM model', type=lstm_model2, required=True) can be written inside statefulv2.py But this raises an invalid argument error.

Is there a neat way to pass methods as arguments in argparse?

The reason for keeping the method outside is to edit it for trying differeny functions since GoogleColab does not provide separate editor for changing model in a separate file.

If you are trying to call a function based on argparse stackoverflow.com/questions/27529610/… then see the related answer there. — ayman
– ayman, Commented Aug 5, 2019 at 19:24
It's probably easier to just write your function into a module file that you load based on a keyword argument. — Linuxios
– Linuxios, Commented Aug 5, 2019 at 19:31
I mentioned at the end of my post how that is a problem on google colab. I will have to upload the file everytime i modify it since colab doesn't have editors — CS101
– CS101, Commented Aug 5, 2019 at 20:05
On a different note, is there no way to pass a method as an argument during a script execution? — CS101
– CS101, Commented Aug 5, 2019 at 20:06
What get's passed through the command line and an argparse (or similar parser) is just a string, a name. Not an object or function. Typically when you want a new function in a script, you import it - via another .py file. — hpaulj
– hpaulj, Commented Aug 5, 2019 at 20:09

Vedang Waradpande · Accepted Answer · 2019-08-07 05:16:51Z

2

It's best that you don't try to pass arguments like that. Stick to the basic types. An alternative would be to store the different models in a file like models.py, such as:

def lstm_model1(...):
    # Definition of model1

def lstm_model2(...):
    # Definition of model2

and so on.

In statefulv2.py you can have:

import models
import argparse

parser = ...
parser.add_argument('-m', "--model", help='LSTM model', 
                      type=str, choices=['model1', 'model2'], required=True)

model_dict = {'model1': models.lstm_model1(...),
              'model2': models.lstm_model2(...)}

args = parser.parse_args()

my_model = model_dict[args.model]

EDIT: If saving model to file is not allowed.

In case you absolutely have to find a workaround, you can save the code in a buffer file which can be read into statefulv2.py as an argument of type open. Here is a demo:

In your Colab cell, you write the function code in a string:

def save_code_to_file():
    func_string = """
def lstm_model3():
    print ("Hey!")
"""
    with open("buffer", "w") as f:
        f.write(func_string)

In statefulv2.py, you can have:

import argparse

parser = argparse.ArgumentParser()

parser.add_argument('-m', "--model", help='LSTM model', type=open, required=True)

args = parser.parse_args()

lines = args.model.readlines()

model_def = ""
for line in lines:
    model_def += line

exec(model_def)

lstm_model3()

Output (this is in iPython, change it accordingly for Colab):

In [25]: %run statefulv2.py -m buffer                                                  
Hey!

edited Aug 7, 2019 at 5:16

answered Aug 5, 2019 at 19:36

Vedang Waradpande

6821 gold badge6 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

CS101 Over a year ago

Yes, this works but the issue is that my local workstation does not have resources for running larger models. And while colab does that, it does not let us edit files on the go. In this case I will have to reupload models.py everytime

Vedang Waradpande Over a year ago

There are two workarounds I can think of: Running statefulv2.py in a Colab cell and writing the function to a temporary string and then reading as an open type in statefulv2.py. I will edit my answer.

Collectives™ on Stack Overflow

Pass a function/method to a python script

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related