connect to url in python

Question

i am trying to connect to a url with a username and password with the following code:

urllib.request.urlopen("http://username:[email protected]...", None)

but i'm getting

urllib.error.URLError: urlopen error [Errno 11003] getaddrinfo failed

anyone know what's up?

What do you mean by URL with username and password? Do you mean a page that will return "401 Unauthorized". — Haozhun
– Haozhun, Commented Jan 16, 2011 at 15:57
BTW, You need to use "@Gene" if you want me be notified because of your reply. (I saw this reply because I anticipated you don't know you need this) — Haozhun
– Haozhun, Commented Jan 16, 2011 at 16:07
@rach: What SilentGhost means is this: Go back to one of your old questions. Decide which answer helped you most. Click on the outline of a checkmark next to that answer. Repeat for all of your questions. — John
– John, Commented Jan 16, 2011 at 23:00
@rach: That's a strange error. I get "Nonnumeric port". In any case it's clear that urllib doesn't accept the username:password@hostname syntax. So, What Gene said. — Lennart Regebro
– Lennart Regebro, Commented Jan 16, 2011 at 23:18

Haozhun · Accepted Answer · 2011-01-16 16:37:08Z

5

I'm sorry. I didn't notice you are using py3k.
See urllib.request - FancyURLopener. I personally don't know py3k very well.
Basically, you need to subclass urllib.request.FancyURLopener, override prompt_user_passwd(host, realm), and then call YourClass.urlopen(url).

Below is for py2

This is what you want, urllib2 - Basic Authentication
Below is the code from that page, just in case some day the link rot.

# create a password manager
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()

# Add the username and password.
# If we knew the realm, we could use it instead of None.
top_level_url = "http://example.com/foo/"
password_mgr.add_password(None, top_level_url, username, password)

handler = urllib2.HTTPBasicAuthHandler(password_mgr)

# create "opener" (OpenerDirector instance)
opener = urllib2.build_opener(handler)

# use the opener to fetch a URL
opener.open(a_url)

# Install the opener.
# Now all calls to urllib2.urlopen use our opener.
urllib2.install_opener(opener)

edited Jan 16, 2011 at 16:37

answered Jan 16, 2011 at 16:08

Haozhun

6,5313 gold badges32 silver badges51 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Haozhun Over a year ago

@SlientGhost Thank you. I didn't notice that. I've update it soon.

Haozhun Over a year ago

@rach You're welcome. But I guess you'll have to figure out how it works on py3k. It's not as straightforward as py2 is. BTW, in this case, you can omit @Gene because I'm the owner of this answer and will automatically be notified.

Lennart Regebro Over a year ago

It is exactly as straightforward, but instead of urllib2 you use urllib.request.

ulidtko · Accepted Answer · 2011-01-16 16:12:13Z

2

You should use urllib.request.HTTPBasicAuthHandler for HTTP authentication.

HTTP doesn't handle authentication in the user:password@host way.

answered Jan 16, 2011 at 16:12

ulidtko

15.8k10 gold badges59 silver badges93 bronze badges

Comments

jfs · Accepted Answer · 2011-01-16 23:41:12Z

1

If you can install 3rd party libraries then httplib2 is both easier to use and more powerful alternative to urllib.request:

import httplib2

h = httplib2.Http("/path/to/cache-directory")
h.add_credentials(username, password)
response, content = h.request(url)
assert response.status == 200

answered Jan 16, 2011 at 23:41

jfs

417k210 gold badges1k silver badges1.7k bronze badges

Collectives™ on Stack Overflow

connect to url in python

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related