HTTP 403 error when trying to download a file from URL using Python

Question

I'm trying to download a file from the URL -> https://www.microsoft.com/en-us/download/confirmation.aspx?id=56519

I can manually download the file by accessing the URL via a browser and the file gets automatically saved onto the local machine in the Downloads folder. (The file is in JSON format)

However, I need to achieve this using a Python script. I tried using urllib.request & wget, but in both cases I keep getting the error -

    urllib.request.urlretrieve(url, path)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 247, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

Kevin Lu · Accepted Answer · 2021-04-02 08:35:48Z

3

Is there a workaround to this? Dealing with dynamic changes ?

You could try the following script to get the download url and download the json file:

import requests
import re
import urllib.request

rq= requests.get("https://www.microsoft.com/en-us/download/confirmation.aspx?id=56519")
 
t = re.search("https://download.microsoft.com/download/.*?\.json", rq.text )
 


a= t.group()

print(a)

path = r"$(Build.sourcesdirectory)\agent.json"
urllib.request.urlretrieve(a, path)

Result:

answered Apr 2, 2021 at 8:35

Kevin Lu

36.8k4 gold badges36 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

scalloty · Accepted Answer · 2021-04-01 11:43:37Z

1

Python 3

import urllib.request, json 
with urllib.request.urlopen("https://download.microsoft.com/download/7/1/D/71D86715-5596-4529-9B13-DA13A5DE5B63/ServiceTags_Public_20210329.json") as url:
    data = json.loads(url.read().decode())
    print(data)

answered Apr 1, 2021 at 11:43

scalloty

1186 bronze badges

1 Comment

SD4 Over a year ago

Hey! So the filename - "ServiceTags_Public_20210329.json" is dynamic. It changes every week with the date appended to the end of the filename like above. Is there a workaround to this? Dealing with dynamic changes ? Thanks!

Collectives™ on Stack Overflow

HTTP 403 error when trying to download a file from URL using Python

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related