Python JSON formatter

Question

I am new to python. Just starting to experiment it with small programs. I saw this question where :

Input will be of json can be of this format :

'[ ["a","b","c"], [1,2,null], [null,3,4], [5,null,6] ]'

or :

'[ { "a":1, "b":2 }, { "b":3, "c":4 }, { "c":6, "a":5 } ]'

We should convert it into :

output = '{ "a": [1,null,5], "b": [2,3,null], "c": [null,4,6] }'

So far I can think of is checking each element and appending to the result. Is there any easy or better way to do this in python. Please enlighten me.

@IronFist seems like null is valid in json. If you call json.loads on that string it converts it to None. Presumably it also works the other way round. — Paul Rooney
– Paul Rooney, Commented Jan 21, 2016 at 7:05
@PaulRooney .. correct .. :) ..thought it was in the dictionary not in the json output...didn't pay attention to that — Iron Fist
– Iron Fist, Commented Jan 21, 2016 at 7:08
There are two tasks involved here: Firstly, read the JSON. Secondly, change the structure of the resulting data to resemble the desired output. Which task is giving you problems? — Ulrich Eckhardt
– Ulrich Eckhardt, Commented Jan 21, 2016 at 7:24

Iron Fist · Accepted Answer · 2016-01-21 08:23:26Z

2

Use defaultdict from collections module, this way:

>>> import json
>>> s = '[ { "a":1, "b":2 }, { "b":3, "c":4 }, { "c":6, "a":5 } ]'
>>> 
>>> dic = json.loads(s)
>>> dic
[{'a': 1, 'b': 2}, {'b': 3, 'c': 4}, {'a': 5, 'c': 6}]
>>> kys = set(k for sub_d in d for k in sub_d) #creates uniques keys of dictionary d
>>> kys
{'a', 'b', 'c'}
>>>
>>> from collections import defaultdict
>>> my_dict = defaultdict(list)
>>> for d in dic:
        for k in kys:
            my_dict[k].append(d.get(k, None))
>>> my_dict
defaultdict(<class 'list'>, {'a': [1, None, 5], 'b': [2, 3, None], 'c': [None, 4, 6]})

As for the other situation:

>>> s = '[ ["a","b","c"], [1,2,null], [null,3,4], [5,null,6] ]'
>>> d = json.loads(s)
>>> d
[['a', 'b', 'c'], [1, 2, None], [None, 3, 4], [5, None, 6]]
>>> my_dict = dict(zip(d[0], zip(*d[1:])))
>>> my_dict
{'a': (1, None, 5), 'b': (2, 3, None), 'c': (None, 4, 6)}

If you don't want tuples as values, then:

>>> my_dict = defaultdict(list)
>>> for k,v in zip(d[0], zip(*d[1:])):
        my_dict[k].extend(v)

Finally, to group both cases into one function:

import json
from collections import defaultdict

def parse_data(data):
    data = json.loads(data) 
    my_dict = defaultdict(list)
    if isinstance(data[0], list):
        for k,v in zip(data[0], zip(*data[1:])):
            my_dict[k].extend(v)
    elif isinstance(data[0], dict):
        kys = set(k for sub_d in data for k in sub_d)
        for d in data:
            for k in kys:
                my_dict[k].append(d.get(k, None))
    return my_dict

s1 = '[ ["a","b","c"], [1,2,null], [null,3,4], [5,null,6] ]' 
d1 = parse_data(s1)
s2 = '[ { "a":1, "b":2 }, { "b":3, "c":4 }, { "c":6, "a":5 } ]'
d2 = parse_data(s2)

edited Jan 21, 2016 at 8:23

answered Jan 21, 2016 at 7:18

Iron Fist

11k2 gold badges21 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Sarah Over a year ago

Can you explain this line : set(k for sub_d in d for k in sub_d) What is sub_d here ?

Iron Fist Over a year ago

set will return unique element from any collections or list and because d is list of dictionaries, I had to use set comprehension for that, as if to say: for every sub_d (every dictionary in list) then for every k in that sub_d ...

Iron Fist Over a year ago

@Sarah .. Check my edit as I've grouped both situations into one function.

PatNowak · Accepted Answer · 2016-01-21 07:38:27Z

1

Try this with Python 3:

def get_elements(json_txt):
import json
arr = json.loads(json_txt)
new = {}
list_of_keys = []
list_of_keys_from_dicts = [list(elem.keys()) for elem in arr]
# getting keys from json
for keys in list_of_keys_from_dicts:
    for key in keys:
        if key not in list_of_keys:
            list_of_keys.append(key)
for key in list_of_keys:
    new[key] = []
for element in arr:
    for key in list_of_keys:
        if key in element:
            new[key].append(element[key])
        else:
            new[key].append(None)
return json.dumps(new)

edited Jan 21, 2016 at 7:38

answered Jan 21, 2016 at 7:32

PatNowak

5,8221 gold badge27 silver badges33 bronze badges

Comments

dhana · Accepted Answer · 2016-01-21 07:23:44Z

It will works for both formates

import json


def convert_format(json_data):
    convert_to_py_obj = json.loads(json_data)
    new_format = dict()
    if isinstance(convert_to_py_obj, list) and len(convert_to_py_obj) > 1:
        index_0_keys = convert_to_py_obj[0]
        if isinstance(index_0_keys, list):
            for i, key in enumerate(index_0_keys):
                new_format[key] = []
                for sub_list in convert_to_py_obj[1:]:
                    new_format[key].append(sub_list[i])
        elif isinstance(index_0_keys, dict):
            for sub_dict in convert_to_py_obj:
                for key, val in sub_dict.iteritems():
                    if key in new_format:
                        new_format[key].append(val)
                    else:
                        new_format[key] = [val]
                none_keys = set(new_format.keys()) - set(sub_dict.keys())
                for key in none_keys:
                    new_format[key].append(None)
    return json.dumps(new_format)

Collectives™ on Stack Overflow

Python JSON formatter

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related