Python: Replacing url with title

Question

I have written this code to replace urls with their titles. It does replace urls with titles as required but it prints their titles in next line.

twfile.txt contains these lines:

link1 http://t.co/HvKkwR1c
no link line

Output tw2file:

link1
Instagram
no link line

but i want output in this form:

link1 Instagram
no link line

What should i do?

My Code:

from bs4 import BeautifulSoup
import urllib

output = open('tw2file.txt','w')

with open('twfile.txt','r') as inputf:
    for line in inputf:
        try:
            list1 = line.split(' ')
            for i in range(len(list1)):

                if "http" in list1[i]:
                    ##print list1[i]
                    response = urllib.urlopen(list1[i])
                    html = response.read()
                    soup = BeautifulSoup(html)
                    list1[i] = soup.html.head.title
                    ##print list1[i]


                    list1[i] = ''.join(ch for ch in list1[i])
                else:
                    list1[i] = ''.join(ch for ch in list1[i])
            line = ' '.join(list1)
            print line
            output.write(line)
        except:
            pass


inputf.close()
output.close()

emesday · Accepted Answer · 2014-07-20 09:28:56Z

Try this code: (see here, here, and here)

from bs4 import BeautifulSoup
import urllib

with open('twfile.txt','r') as inputf, open('tw2file.txt','w') as output:
    for line in inputf:
        try:
            list1 = line.split(' ')
            for i in range(len(list1)):
                if "http" in list1[i]:
                    response = urllib.urlopen(list1[i])
                    html = response.read()
                    soup = BeautifulSoup(html)
                    list1[i] = soup.html.head.title
                    list1[i] = ''.join(ch for ch in list1[i]).strip() # here
                else:
                    list1[i] = ''.join(ch for ch in list1[i]).strip() # here
            line = ' '.join(list1)
            print line
            output.write('{}\n'.format(line))  # here
        except:
            pass

BTW, you are using Python 2.7.x +, two opens expressed in the same with clause. Also their closes are unnecessary.

Gio · Accepted Answer · 2014-04-28 08:36:47Z

1

Regarding the content written to a file

fileobject = open("bar", 'w' )
fileobject.write("Hello, World\n") # newline is inserted by '\n'
fileobject.close()

Regarding console output

Change print line to print line,

Python writes the '\n' character at the end, unless the print statement ends with a comma.

edited Apr 28, 2014 at 8:36

answered Apr 28, 2014 at 8:12

Gio

3,3502 gold badges27 silver badges57 bronze badges

3 Comments

Nabila Shahid Over a year ago

it's not affecting the output

Gio Over a year ago

why are you printing 2 times ? print line and output.write(line)?

emesday Over a year ago

print seems for console. the other seems for file

Collectives™ on Stack Overflow

Python: Replacing url with title

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related