Getting Value from Response from Requests on python

Question

I am using requests and I'm trying to get the response (as seen on Network tab on chrome)

url = r"https://wwww.mylocal.com/345523"
authKey = base64.b64encode("testu:pwd$u")
headers = {"Content-Type":"application/json", "Authorization":"Basic " + "YWRtaW46OWJKTmp5WUxKc1A0dlJ1dTd2Mk0"}
data = { "id":444}
r = requests.get(url, headers=headers, params = data)
print r.content

But I keep getting the HTML of the page and I want to get actual response. When using r.content I get a message that object cannot be decoded, which makes sense since r.content retrieves the source code instead of actual response. Is there a way to get that? The response I see on network tab is similar to:

{"search_results": {"@xmlns": "http://mylocal.com/345523", "ColorsInfo": {"bgColor": "552448"}}

And what I want to retrieve is the value of bgcolor but on r.content I get the Html content of the page.

<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
...and so on

PyNaobie · Accepted Answer · 2015-07-15 17:25:46Z

1

solved it! Parameter has to be a string so correct code:

url = r"https://wwww.mylocal.com/345523"
authKey = base64.b64encode("testu:pwd$u")
headers = {"Content-Type":"application/json", "Authorization":"Basic " + "YWRtaW46OWJKTmp5WUxKc1A0dlJ1dTd2Mk0"}
data = { "id":"444"}
r = requests.get(url, headers=headers, params = data)
print r.content

answered Jul 15, 2015 at 17:25

PyNaobie

2151 gold badge4 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

berto · Accepted Answer · 2015-07-14 07:03:44Z

0

My first guess is that the think you are getting is a login form because you have not properly logged into the site.

Can you verify this is the case by reading through the html or writing the html into a file and opening it up in the browser.

If my suspicions are correct look at requests session object (http://docs.python-requests.org/en/latest/user/advanced/), post the username and password via form as you would in the browser and then request the data above.

answered Jul 14, 2015 at 7:03

berto

8,5734 gold badges27 silver badges22 bronze badges

3 Comments

PyNaobie Over a year ago

Thanks for the answer, but credentials are ok, the html on response is the content of the homepage so no login form

berto Over a year ago

In Chrome's Network tab can you take a screenshot of the Request Headers section in the Headers tab of the request you're making from Python?

berto Over a year ago

Can you update your code to use requests' auth=(username, password) kwarg rather than placing it in the header yourself.

Collectives™ on Stack Overflow

Getting Value from Response from Requests on python

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related