Repeat Python function call on exception?

Question

Hey everybody I'm working on a data scraping project and I'm looking for a clean way to repeat a function call if an exception is raised.

Pseudo-code:

try:
    myfunc(x)
except myError:
    ###try to call myfunc(x) again Y number of times, 
        until success(no exceptions raised) otherwise raise myError2

I realize this isn't best practice at all but I'm working through a number of different code/network layers that aren't reliable and I can't realistically debug them.

Right now I'm accomplishing this with a huge set of try\except blocks and it's making my eyes bleed.

Elegant ideas anyone?

This is a situation where a goto would be incredibly useful. — Rafe Kettler
– Rafe Kettler, Commented Jan 22, 2011 at 6:54
Generally, and especially in case of a networking task an exponential backoff should be used to find appropriate rate. Retrying right away may lead to waste of resources. The idea is illustrated in this Python decorator recipe wiki page. — saaj
– saaj, Commented Mar 11, 2016 at 15:51

user470379 · Accepted Answer · 2011-01-22 07:16:14Z

13

To do precisely what you want, you could do something like the following:

import functools
def try_x_times(x, exceptions_to_catch, exception_to_raise, fn):
    @functools.wraps(fn) #keeps name and docstring of old function
    def new_fn(*args, **kwargs):
        for i in xrange(x):
            try:
                return fn(*args, **kwargs)
            except exceptions_to_catch:
                 pass
        raise exception_to_raise
    return new_fn

Then you just wrap the old function in this new function:

#instead of
#risky_method(1,2,'x')
not_so_risky_method = try_x_times(3, (MyError,), myError2, risky_method)
not_so_risky_method(1,2,'x')

#or just
try_x_times(3, (MyError,), myError2, risky_method)(1,2,'x')

edited Jan 22, 2011 at 7:16

answered Jan 22, 2011 at 7:03

user470379

4,88719 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Fred Nurk Over a year ago

Use functools.wraps instead of changing __name__ yourself.

Haipa Over a year ago

Cool. I actually ended up using peter-hoffmann.com/2010/retry-decorator-python.html but it's pretty much the same idea.

Ming-Tang · Accepted Answer · 2011-01-22 07:13:38Z

8

Use a loop

i = 0
while True:
  try: myfunc(x); break;
  except myError:
    i = i + 1;
    # print "Trying again"
    if i > 5: raise myError2;

edited Jan 22, 2011 at 7:13

answered Jan 22, 2011 at 6:56

Ming-Tang

17.7k8 gold badges40 silver badges78 bronze badges

Comments

primroot · Accepted Answer · 2011-01-22 07:10:32Z

4


for x in xrange(num_retries):
    try:
        myFunc()
    except MyError, err:
        continue
        #time.sleep(1)
    err = None
    break
if err:
    raise MyError2
#else:
#    print "Success!"

edited Jan 22, 2011 at 7:10

answered Jan 22, 2011 at 7:04

primroot

1,7061 gold badge15 silver badges17 bronze badges

Comments

binithb · Accepted Answer · 2017-06-23 13:59:08Z

To raise Exception as usual after n retries

from functools import wraps

def retry(times):
    """
    Decorator to retry any functions 'times' times.
    """
    def retry_decorator(func):
        @wraps(func)
        def retried_function(*args, **kwargs):
            for i in range(times - 1):
                try:
                    func(*args, **kwargs)
                    return
                except Exception:  
                    pass
            func(*args, **kwargs)

        return retried_function

    return retry_decorator


# test

attempts = 3

@retry(4)
def function_that_raises_error():
    global attempts
    if 0 < attempts:
        print("fail")
        attempts -= 1
        raise Exception

    print("pass")

function_that_raises_error()

Jochen Ritzel · Accepted Answer · 2011-01-22 14:59:51Z

1

I like to do these problems with recursion:

def tryfor(times, on_failure, excepts, func, *args, **kwargs):
    if times < 1:
        raise on_failure()
    try:
        return func(*args, **kwargs)
    except excepts:
        return tryfor(times-1, on_failure, excepts, func, *args, **kwargs)


tryfor(3, PermanentException, (SomeError,), dostuff,1,2)

answered Jan 22, 2011 at 14:59

Jochen Ritzel

108k33 gold badges205 silver badges195 bronze badges

1 Comment

Nakilon Over a year ago

Does it tail-optimize?

Elalfer · Accepted Answer · 2011-01-22 06:56:36Z

0

Try following snippet:

while True:
    try:
        func()
        break
    except:
        print "Error. Gonna try again"

But it is better to limit the number of retries.

answered Jan 22, 2011 at 6:56

Elalfer

5,39424 silver badges25 bronze badges

1 Comment

AndiDog Over a year ago

except: is not a good idea in most (but not all) cases. I'd rather use except Exception:. And this of course assumes that func() will terminate at all.

inspectorG4dget · Accepted Answer · 2011-01-22 07:00:07Z

0

success = False
attempts = 0
while not success and attempts < 10: # or however many times you want to attempt
    try:
        functionCall()
        success = True
    except:
        i += 1
if not success:
    raise functionCallFailedError

Hope this helps

answered Jan 22, 2011 at 7:00

inspectorG4dget

115k30 gold badges159 silver badges253 bronze badges

Collectives™ on Stack Overflow

Repeat Python function call on exception?

7 Answers 7

2 Comments

Comments

Comments

Comments

1 Comment

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

7 Answers 7

2 Comments

Comments

Comments

Comments

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related