json.loads(string) throws json.decoder.JSONDecodeError when trying to convert string to dictionary

Question

I'm using the requests library to make an api call. The json response is then formatted as a string and sent as part of a result to my server as shown by the code snippet:

def get_and_send(url, method):
     resp = requests.request(url=url, method=method, **kwargs)
     result = f'{{ "status_code":{resp.status_code}, "content":{resp.json()} }}'
     send_to_server(result)

I intend to convert this result back to a dictionary object from the string result on the server.
The problem I have is that when I use json.loads(result) to convert the string to dictionary object, it throws the following error

Exception in thread Thread-2: Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
File "/home/adipster/PycharmProjects/ScriptBackbone/ts_server/agent_thread.py", line 39, in run
    resp_data = self._task_formatter.format_response(response) # Formats the response
File "/home/adipster/PycharmProjects/ScriptBackbone/utils/task_formatter.py", line 26, in format_response
    response = self.get_dict_response(response.decode().strip())
File "/home/adipster/PycharmProjects/ScriptBackbone/utils/task_formatter.py", line 36, in get_dict_response
    raise exp
File "/home/adipster/PycharmProjects/ScriptBackbone/utils/task_formatter.py", line 34, in get_dict_response
    return json.loads(response)
File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.6/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 32 (char 31)

I understand that the error is because all my keys have to be in double quotes which is not the case when resp.json() is formatted to a string. Please can someone help out on how I could ensure that all the keys of my dictionary object are in double quotes? Or any other alternative to this will be welcomed. Thanks

We need to see the specific response to speak to why it's not valid JSON. — Charles Duffy
– Charles Duffy, Commented Mar 10, 2019 at 23:51
...or you could create it with json.dumps(), so it would be guaranteed to be valid JSON. That is: result = json.dumps({"status_code": resp.status_code, "content": resp.json()}) (if resp.json() returns a native Python data structure, that is). — Charles Duffy
– Charles Duffy, Commented Mar 10, 2019 at 23:52
@CharlesDuffy, I think you make a point there. I'll give your suggestion a go. — Cels
– Cels, Commented Mar 10, 2019 at 23:57
@CharlesDuffy you should have posted that as an answer. It works like charm — Cels
– Cels, Commented Mar 11, 2019 at 0:01

Henry · Accepted Answer · 2019-03-10 23:58:54Z

5

The issue is, as you point out, that if you just do string interpolation (with an f-string, for example), the quotes for strings in the JSON object will be single quotes, but JSON format requires double quotes.

To fix this you can use json.dumps which takes a JSON object (in Python) and converts it to a properly formatted JSON string. Edit thanks to Charles Duffy: You can avoid the f-string entirely by just creating the whole result_data object as a dictionary, then converting it to JSON all at once using json.dumps.

For example:

import json

def get_and_send(url, method):
    resp = requests.request(url=url, method=method, **kwargs)
    result_data = {
        "status_code": resp.status_code,
        "content": resp.json(),
    }
    result = json.dumps(result_data)
    send_to_server(result)

edited Mar 10, 2019 at 23:58

answered Mar 10, 2019 at 23:54

Henry

15.9k7 gold badges51 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Charles Duffy Over a year ago

If we want result to be JSON, why use an f-string at all, rather than letting json.dumps() format the top-level as well?

Henry Over a year ago

@CharlesDuffy oh that's better

Collectives™ on Stack Overflow

json.loads(string) throws json.decoder.JSONDecodeError when trying to convert string to dictionary

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related