Python http request not returning json output

Question

When I am using following command, I get proper json response from the url: curl --user user:password -H Accept:application/json -H Content-Type:application/json -X GET <url>

"body": { "items": [ {........}]}

However, when I am trying to do same using python request module

import requests

from requests.packages.urllib3.exceptions import InsecureRequestWarning

requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

user='admin'
password='yanQE3B'
url=<requestUrl>

response = requests.get(url, auth=(user, password) , verify=False)

print ( response.content )  ## prints html content

print ( response.text )     ## prints html content

data = response.json()    ## error json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

when i do request.json() , I get an error json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
when i print request.content - it returns html list with html content. Output of reponse.text and response.content is same. Truncated html as large number of li tags get returned :

<!DOCTYPE html>
<html>
<head>
<title>Data</title>
</head>
<div>
<ul>
<li>
body&nbsp;
<ul>
<li>items&nbsp;<ol>
<li>
<ul>
<li>name&nbsp;........ </li><li>messages&nbsp;<ol>
</ol></li></ul></div></body></html>

cool_stuff_coming · Accepted Answer · 2020-08-07 08:56:16Z

1

headers={'Accept':'application/json'} added to requests.get

answered Aug 7, 2020 at 8:56

cool_stuff_coming

4531 gold badge8 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Gahan · Accepted Answer · 2020-08-07 08:38:19Z

0

It seems that the URL you are trying to get JSON from is by default returning the HTML page only.

From the curl request where you have explicitly passed the content type application/json hence you need to similarly update your request statement as below:

response = requests.get(url, auth=(user, password), verify=False, 'headers': {'Content-Type': 'application/json'})

Note: Above method works only if your URL actually returns the JSON response.

answered Aug 7, 2020 at 8:38

Gahan

4,2234 gold badges26 silver badges47 bronze badges

2 Comments

cool_stuff_coming Over a year ago

i used headers={'Accept':'application/json'} which resolved the issue. Thanks for your hint.

Gahan Over a year ago

@cool_stuff_coming Also, it is preferred to use post method as you are using authentication with username and password!

Collectives™ on Stack Overflow

Python http request not returning json output

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related