How to filter JSON data using Python?

Question

How to convert JSON data from input.json to output.json using Python? In general, what data structures are used for filtering JSON data?

File: input.json

[
{
    "id":1,
    "a":22,
    "b":11
},
{
    "id":1,
    "e":44,
    "c":77,
    "f":55,
    "d":66
},
{
    "id":3,
    "b":11,
    "a":22
},
{
    "id":3,
    "d":44,
    "c":88
}
]

File: output.json

[
{
    "id":1,
    "a":22,
    "b":11,
    "e":44,
    "c":77,
    "f":55,
    "d":66
},
{
    "id":3,
    "b":11,
    "a":22,
    "d":44,
    "c":88
}
]

Any pointers would be appreciated!

Have you considered dictionaries? :D

Adib
– Adib

2016-04-25 20:16:41 +00:00
Commented Apr 25, 2016 at 20:16 — Adib
– Adib, Commented Apr 25, 2016 at 20:16

alecxe · Accepted Answer · 2016-04-25 20:17:38Z

The idea is to:

use json.load() to load the JSON content from file to a Python list
regroup the data by the id, using collections.defaultdict and .update() method
use json.dump() to dump the result into the JSON file

Implementation:

import json
from collections import defaultdict

# read JSON data
with open("input.json") as input_file:
    old_data = json.load(input_file)

# regroup data
d = defaultdict(dict)
for item in old_data:
    d[item["id"]].update(item)

# write JSON data
with open("output.json", "w") as output_file:
    json.dump(list(d.values()), output_file, indent=4)

Now the output.json would contain:

[
    {
        "d": 66,
        "e": 44,
        "a": 22,
        "b": 11,
        "c": 77,
        "id": 1,
        "f": 55
    },
    {
        "b": 11,
        "id": 3,
        "d": 44,
        "c": 88,
        "a": 22
    }
]

Günther Jena · Accepted Answer · 2016-04-25 20:22:50Z

3

from collections import defaultdict

input_list=[{"id":1, ...}, {...}]

result_dict=defaultdict(dict)
for d in input_list:
    result_dict[d['id']].update(d)

output_list=result_dict.values()

result_dict is a default dictionary which uses a dict for every access without a available key. So we iterate through the input_list and update our result_dict with key equals id with the new values from the corresponding dictionary.

The output list is a transformation of the result_dict and uses only its values.

Use the json module to work directly with the json data.

edited Apr 25, 2016 at 20:22

answered Apr 25, 2016 at 20:17

Günther Jena

3,7663 gold badges37 silver badges49 bronze badges

Collectives™ on Stack Overflow

How to filter JSON data using Python?

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related