Read file, extract url and re-write -Python

Question

I'm reading text file in the following format(a.txt).

http://www.example.com/forum/showthread.php?t=779689/images/webcard.jpg 121.10.208.31

Then I need to obtain only the www.example.com part with /images/webcard.jpg 121.10.208.31 and write to the same file or a separate one. In this case I'm writing it to b.txt.

from urlparse import urlparse 
f = open('a.txt','r')
fo = open('b','w')


for line in f:
    fo.write(urlparse(line).netloc+ ' ' + line.split(' ')[1] + ' ' + line.split(' ')[2] + '\n')

the above code gives the following error?How to achieve this?

    Traceback (most recent call last):
  File "prittyprint.py", line 17, in <module>
    fo.write(urlparse(line).netloc+ ' ' + line.split(' ')[1] + ' ' + line.split(' ')[2] + '\n')
IndexError: list index out of range

maybe there are exceptions in your file a.txt? some line might not have this format. Can you print the line where it crashes? — Srikar Appalaraju
– Srikar Appalaraju, Commented Jun 16, 2013 at 6:20

Srikar Appalaraju · Accepted Answer · 2013-06-16 06:22:39Z

3

It could be that there are exceptions in your file a.txt. Some line(s) might not have this format. You can try this -

from urlparse import urlparse 

f = open('a.txt','r')
fo = open('b','w')

for line in f:
    split_line = line.split(' ')
    if len(split_line) >=3:
        fo.write(urlparse(line).netloc+ ' ' + split_line[1] + ' ' + split_line[2] + '\n')
    else:
        print "ERROR: some other line: %s" % (line) #continue on with next line

answered Jun 16, 2013 at 6:22

Srikar Appalaraju

74k55 gold badges221 silver badges265 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Nilani Algiriyage Over a year ago

It prints Error message for all the lines in a.txt?

Srikar Appalaraju Over a year ago

then there is something wrong. Can you show the exact line where the crash happens? are all the lines of the format http://www.example.com/forum/showthread.php?t=779689 /images/webcard.jpg 121.10.208.31 ?

Nilani Algiriyage Over a year ago

Yes,each line is in same format and has three blocks seperated by spaces...I used try: statement? Then it worked?for line in f: split_line = line.split(' ') try: fo.write(urlparse(line).netloc+ ' ' + split_line[1] + ' ' + split_line[2] + '\n') except: pass........but this is not perfect isn’t it?

Srikar Appalaraju Over a year ago

ofcourse not! all you are doing is simply ignoring the error cases since you are doing pass in except:. This makes your program crash proof :)

Nilani Algiriyage Over a year ago

Ok Thanks a lot..I'll try to figure about the problem with my text file.

|

Collectives™ on Stack Overflow

Read file, extract url and re-write -Python

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related