Python requests failed with authorization

Question

I am trying to get some videos downloaded from the website. So when I enter the url in the browser, the webpage will ask me for a username and password with a little window popped out. After I enter them there, the file automatically starts to download. But I got 404 when I tried to use requests.get with an auth parameter to do the same thing.

Code:

url = video_links[0]
print("url: ", url)

filename = os.path.basename(urlparse(url).path)
print('filename: ', filename)

r = requests.get(url, auth=(username, password))
print(r.status_code)

Output:

url:  http://datasets.d2.mpi-inf.mpg.de/movieDescription/protected/avi/0001_American_Beauty/0001_American_Beauty_00.00.51.926-00.00.54.129.avi

filename:  0001_American_Beauty_00.00.51.926-00.00.54.129.avi

404

popped window

I have tried to pass a user-agent in. Still not working.

Code ver2:

headers = {
    'User-Agent': 'Mozilla/something more',
}
with requests.Session() as s:
    r = s.get(url, headers=headers)
    print(r.content)

output:

b'<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>401 Unauthorized</title>\n</head><body>\n<h1>Unauthorized</h1>\n<p>This server could not verify that you\nare authorized to access the document\nrequested.  Either you supplied the wrong\ncredentials (e.g., bad password), or your\nbrowser doesn\'t understand how to supply\nthe credentials required.</p>\n<hr>\n<address>Apache/2.4.38 (Debian) Server at datasets.d2.mpi-inf.mpg.de Port 443</address>\n</body></html>\n'

hi, is HTTPBasicAuth failing? Perhaps set your user agent to a known browser as well — jspcal
– jspcal, Commented Sep 2, 2021 at 19:06
Please post your code and out as text. Screenshots of code can not be accepted on Stack Overflow. — Klaus D.
– Klaus D., Commented Sep 2, 2021 at 19:10
show real URL for this page. How do you send login and password? Maybe browser sends it in different way. OR maybe browser uses some JavaScript for redirect - but requests can't run JavaScript. — furas
– furas, Commented Sep 3, 2021 at 11:35

Invoker · Accepted Answer · 2021-09-05 14:18:29Z

1

Just like jspcal said, everything is working now. Thanks

r = requests.get(url, headers=headers, auth=HTTPBasicAuth('username', 'password'))

edited Sep 5, 2021 at 14:18

answered Sep 5, 2021 at 1:04

Invoker

215 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python requests failed with authorization

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related