Python read page from URL? Better documentation?

Question

I'm having quite a bit of trouble with Python's documentation. Is there anything like the Mozilla Developer Network for it?

I'm doing a Python puzzle website and I need to be able to read the content of the page. I saw the following posted on a site:

import urllib2

urlStr = 'http://www.python.org/'
try:
  fileHandle = urllib2.urlopen(urlStr)
  str1 = fileHandle.read()
  fileHandle.close()
  print ('-'*50)
  print ('HTML code of URL =', urlStr)
  print ('-'*50)
except IOError:
  print ('Cannot open URL %s for reading' % urlStr)
  str1 = 'error!'

print (str1)

It keeps saying that there is no urllib2 module.

The Python documentation says

The urllib module has been split into parts and renamed in Python 3.0 to urllib.request, urllib.parse, and urllib.error. The 2to3 tool will automatically adapt imports when converting your sources to 3.0. Also note that the urllib.urlopen() function has been removed in Python 3.0 in favor of urllib2.urlopen().

I tried importing urllib.request too, but it ssays urllib 2 is defined... WTF is going on here?

Version 3.2.2

Updated. Any other documentation for Python than what's given? — mowwwalker
– mowwwalker, Commented Oct 1, 2011 at 2:08
@Walkerneo: docs.python.org/py3k/library/urllib.request.html — icktoofay
– icktoofay, Commented Oct 1, 2011 at 2:11
Is this the documentation you're using? docs.python.org/py3k — Thomas
– Thomas, Commented Oct 1, 2011 at 2:12
Yes and yes. I've been using that. @icktoofay thanks, that helps. I have to do urllib.request.openurl then. Thanks, put it as an answer and I'll accept it. — mowwwalker
– mowwwalker, Commented Oct 1, 2011 at 2:14

johnsyweb · Accepted Answer · 2011-10-01 02:20:49Z

4

Using urllib.request.open(), as recommended in Dive into Python 3...

Python 3.2.1 (default, Jul 24 2011, 22:21:06) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request
>>> urlStr = 'http://www.python.org/'
>>> fileHandle = urllib.request.urlopen(urlStr)
>>> print(fileHandle.read()[:100])
b'<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtm'

edited Oct 1, 2011 at 2:20

answered Oct 1, 2011 at 2:15

johnsyweb

143k26 gold badges197 silver badges253 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Dixit Singla Over a year ago

Now file fileHandle contains the complete source. How can I use the xpath on fileHandle data to get particular value?

johnsyweb Over a year ago

@DixitSingla: You could use lxml, like in this answer: stackoverflow.com/a/11466033/78845

icktoofay · Accepted Answer · 2011-10-01 03:17:35Z

3

The documentation you were probably referencing was the Python 2 documentation for urllib2. The documentation you should probably be using is the Python 3 documentation for urllib.request.

edited Oct 1, 2011 at 3:17

answered Oct 1, 2011 at 2:16

icktoofay

130k23 gold badges261 silver badges239 bronze badges

Collectives™ on Stack Overflow

Python read page from URL? Better documentation?

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related