Catch error when passing function as a parameter

Question

I use a python script to update ~500 Google Sheets (using some wrappers on the google api) every morning, and find that I need to catch API errors all the time. I wanted to write a generic "try this function, catch any errors" function, and was able to pass in the function and its parameters successfully, but it doesn't seem to catch the errors. Only if I wrap the original function call in a try/catch does it actually handle the error. This works, but it's starting to feel like half of my code is try/catch protocol.

Should this be working? Is there something different I can do so I only need to write my "try protocols" once?

My attempt at a generic catch function:

def try_function(function, teacher_name="teacher", note="unknown when", tab=None):
    check = True
    count = 0
    while check:
        check = False
        try:
           function
        except Exception as e:
            if str(e).find("Quota exceeded")>-1:
                print("API Quota exceeded " + note + ", waiting 100 seconds - " + teacher_name)
                time.sleep(100)
                check = True
                count+=1
            elif str(e).find("The service is currently unavailable")>-1:
                print("API threw 'currently unavailable error' " + note + ", waiting 100 seconds - " + teacher_name)
                time.sleep(100)
                check = True
                count+=1
            elif str(e).find("Read timed out.")>-1:
                print("API threw 'Read timed out.' error " + note + ", waiting 150 seconds - " + teacher_name)
                time.sleep(150)
                check = True
                count+=1
            elif str(e).find("Internal error encountered.")>-1:
                print("API threw 'Internal error encountered' " + note + ", waiting 100 seconds - " + teacher_name)
                time.sleep(100)
                check = True    
                count+=1
            else:
                print("Non-quota error " + note + ", teacher may not have been updated, waiting 250s - " + teacher_name)
                print(e)
                time.sleep(250)
                del spread
                continue
            
        if count > 4:
            print("Tried this 5 times, ending try loop - " + teacher_name)
            del spread
            continue

Called like this: try_function(spread.df_to_sheet(parameters), "name", "note")

Do I have to wrap that last line in order to actually catch an error?

Assuming spread.df_to_sheet(parameters) is the "API call", you can't do this literally as you want. spread.df_to_sheet(parameters) runs before your function is ever entered. You'd need macros to do this. You could change your function call to try_function(lambda: spread.df_to_sheet(parameters), "name", "note") to wrap the code in a 0-arity function (or you could use functools.partial for this as well), then inside your try_function function, you'd change try: function to try: function(). That just delays the function being called until execution is inside the try block. — Carcigenicate
– Carcigenicate, Commented Jul 30, 2021 at 19:44
What exceptions are actually being raised? Matching specific exception types is far better than trying to match error messages. — chepner
– chepner, Commented Jul 30, 2021 at 20:19
Both comments helpful, thank you! The lambda: + function() edit did what I needed. And the exceptions are mostly generic APIErrors, with a couple exceptions, but I've edited my code to match that first and am dealing with their unique messages below. — Luke Allpress
– Luke Allpress, Commented Aug 3, 2021 at 17:50

Samwise · Accepted Answer · 2021-07-30 20:29:27Z

2

You need to actually call the function inside your try block, and the thing you pass in has to be an actual function (not the result of already having called a function). Simplifying down the exception handling into a generic retry for the sake of making the example simpler (note that in real life you would typically handle different types of exceptions by matching on the type of the exception rather than trying to parse its string representation), it would look like this:

def try_function(function, teacher_name="teacher", note="unknown when"):
    count = 0
    while True:
        try:
            function()
            return
        except Exception as e:
            print(f"Error: {e} ({note}), waiting 100s and retrying...")
            count += 1
            time.sleep(100)
        if count > 4:
            print(f"Tried this 5 times, ending try loop - {teacher_name}")
            return

try_function(lambda: spread.df_to_sheet(parameters), "name", "note")

Note that the lambda expression creates a function with no arguments that will invoke spread.df_to_sheet when it's called (which will happen inside your try block). This is the same as writing:

def to_sheet_fn():
    spread.df_to_sheet(parameters)

try_function(to_sheet_fn, "name", "note")

edited Jul 30, 2021 at 20:29

answered Jul 30, 2021 at 20:23

Samwise

72.1k3 gold badges36 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Luke Allpress Over a year ago

Including lambda: in the call and actually calling the function within my try_function did the trick, thanks so much. I also greatly appreciate the nudge to match the actual exceptions; it's been hard to find useful guidance on that online.

Collectives™ on Stack Overflow

Catch error when passing function as a parameter

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related