Circumventing Python errors in a script

Question

I have a large file containing thousands of links. I've written a script calling each link line-by-line and performing various analyses on the respective webpage. However, sometimes it is the case that the link is faulty (article removed from website, etc), and my whole script just stops at that point.

Is there a way to circumvent this problem? Here's my (pseudo)code:

for row in file:
    url = row[4]
    req=urllib2.Request(url)
    tree = lxml.html.fromstring(urllib2.urlopen(req).read())
    perform analyses
    append analyses results to lists
output data

I have tried

except:
    pass

But it royally messes up the script for some reason.

Vincent Beltman · Accepted Answer · 2014-10-01 15:07:09Z

2

Works for me:

for row in file:
    url = row[4]
    try:
        req=urllib2.Request(url)
        tree = lxml.html.fromstring(urllib2.urlopen(req).read())
        perform analyses
        append analyses results to lists
    except URLError, e:
        pass
output data

answered Oct 1, 2014 at 15:07

Vincent Beltman

2,11216 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

David Robinson Over a year ago

Do you mean to put the try/except block inside the for loop? Otherwise it will stop at that line, exactly the problem the OP wants to avoid

Vincent Beltman Over a year ago

Oh god its getting late. I'm not thinking 100% Thank you for the corection

user2097159 · Accepted Answer · 2014-10-01 15:16:12Z

0

Try block is the way to go:

for row in file:
url = row[4]
    try:
        req=urllib2.Request(url)
        tree = lxml.html.fromstring(urllib2.urlopen(req).read())
    except URLError, e:
        continue
    perform analyses
    append analyses results to lists
output data

Continue will allow you to skip any unnecessary computation after the url check and restart at the next iteration of the loop

answered Oct 1, 2014 at 15:16

user2097159

8925 silver badges14 bronze badges

Collectives™ on Stack Overflow

Circumventing Python errors in a script

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related