Python 2.7: Put extracted data from URL in a CSV file

Question

This is my first time using Python 2.7, I do like it. However, I am trying to figure out how I can put extracted data from the URL into a CSV file. I found this tutorial, but when I run my script:

# import libraries
import csv
import urllib2
from bs4 import BeautifulSoup

# specify the url
quote_page = 'http://www.bkfrem.dk/default.asp?id=19'

# query the website and return the html to the variable ‘page’
page = urllib2.urlopen(quote_page)

# parse the html using beautiful soup and store in variable soup
soup = BeautifulSoup(page, 'html.parser')

# create CSV file
csvfile = csv.writer(open('firsteam.csv', 'w'))
csvfile.writerow(["Name", "Position"])

# take out the <div> of name and get its value
items = soup.find_all('div', attrs={'class': 'visTruppenContainer'})

for i in range(len(items)):

    playerInfo = items[i].getText(separator=u' ')
    imageURL = items[1].find('img')['src']
    csvfile.writerow([playerInfo, imageURL])
    print (playerInfo)
    print (imageURL)

I get this error:

Traceback (most recent call last):
  File "C:/Users/User/Desktop/script2.py", line 26, in <module>
    csvfile.writerow([playerInfo, imageURL])
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe5' in position 27: ordinal not in range(128)

What am I doing wrong? Do I have to convert the data before writing to the CSV file?

Yes in csvfile.writerow([playerInfo, imageURL]) (I can't try it because when I convert this to Python 3 it works) so I can't tell you where exactly but it is certainly in one of those two — Xantium
– Xantium, Commented Feb 11, 2018 at 1:03
Yes, now I don't get the error anymore, but the CSV file is empty :( — Bab
– Bab, Commented Feb 11, 2018 at 1:25

Stephen Rauch · Accepted Answer · 2018-02-11 01:06:35Z

2

You will need to encode the playerInfo like:

Code:

csvfile.writerow([playerInfo.encode('utf-8'), imageURL])

Results:

1. Marco Brylov Position: Målmand Højde: 191 Vægt: 92 Født: 21-11-1995
http://www.bkfrem.dk/images/spillere/02_mikkel_andersson.jpg
2. Mikkel Andersson Position: Midtbane Højde: 170 Vægt: 67 Født: 17-03-1990
http://www.bkfrem.dk/images/spillere/02_mikkel_andersson.jpg
3. Casper Andersen Position: Midtstopper Højde: 190 Vægt: 90 Født: 04-08-1982
http://www.bkfrem.dk/images/spillere/02_mikkel_andersson.jpg
...

answered Feb 11, 2018 at 1:06

Stephen Rauch♦

50.1k32 gold badges118 silver badges143 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Bab Over a year ago

Thank you alot. When I open the csv file, it's, for some reason, empty

Collectives™ on Stack Overflow

Python 2.7: Put extracted data from URL in a CSV file

1 Answer 1

Code:

Results:

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Code:

Results:

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related