Catch Exception in for Loop Python

Question

I have the below for loop:

for batch in loader:
    # do something with batch
    ...

My loop sometimes fails, while extracting the batch from the loader. What I want to do is something similar to snippet below, but I would like to be able to continue the loop on the next value, rather than skip the rest of the values.

error_idxs = [] 

try:
    for i,batch in enumerate(loader):
        # do something with batch
        ...
except:
    error_idxs.append(i)

The problem with the method above is that it exits out of the loop as soon as an exception happens rather than continuing with the next batch.

Is there a way to continue looping at the next batch?

just put the try/except inside the loop instead of outside the loop — Nullman
– Nullman, Commented Jan 7, 2021 at 11:05
If the loader fails how could you ask it again ? And so when stopping ? — azro
– azro, Commented Jan 7, 2021 at 11:07
When the loader fail, you can ask it again after that error ? quite strange — azro
– azro, Commented Jan 7, 2021 at 11:07
I am trying to capture the error in the loop part of the code. So my error occurs while extracting the batch. — Abed Merii
– Abed Merii, Commented Jan 7, 2021 at 11:10

sunnytown · Accepted Answer · 2021-01-07 11:28:16Z

5

error_idxs = []
i = 0
while True:
    try:
        batch = next(loader)
        # do something
    except StopIteration:
        break
    except Exception:
        error_idxs.append(i)
    finally:
        i += 1

Edit: Corrected StopIterationError to StopIteration and removed continue

edited Jan 7, 2021 at 11:28

answered Jan 7, 2021 at 11:11

sunnytown

2,0661 gold badge9 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Abed Merii Over a year ago

I think that might just do it. May I ask what the StopIterationError is for?

Abed Merii Over a year ago

Is it to exit if the iterator runs out of values? @sunnytown

costaparas Over a year ago

Yes, next will raise a StopIteration exception when the iterator is exhausted.

sunnytown Over a year ago

StopIteration is raised when the iterable (so the loader) runs out of new values. A for loop automatically catches the StopIteration behind the scenes and uses it to know when it should stop. The while loop doesn't, since it runs forever, therefore we gotta catch it ourselves.

sunnytown Over a year ago

Btw. if you know the exact exception that is raised when your loader has a connection problem, then I recommend you write that error in instead of Exception. If you don't do that, it will catch all possible exceptions, which is something you want to avoid.

Jaysmito Mukherjee · Accepted Answer · 2021-01-07 11:10:33Z

0

You may use while loop instead.

Here it will be extracted inside loop so exception can be caught inside loop and handled and continued with rest!

error_idxs = [] 

i = -1
while i < len(loader) -1:
    try:
        i = i + 1
        batch = loader[i]
        do something witth batch
        ...
    except:
        error_idxs.append(i)

answered Jan 7, 2021 at 11:10

Jaysmito Mukherjee

1,5522 gold badges12 silver badges33 bronze badges

2 Comments

Fauxcuss Over a year ago

Why calculate the length of loader every iteration of the loop? It's not changing in your example.

Jaysmito Mukherjee Over a year ago

@Fauxcuss thats not exactly the point of focus here, and yea caching it in a variable would definitely be a lot better

Collectives™ on Stack Overflow

Catch Exception in for Loop Python

2 Answers 2

5 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related