python parse Json for all null values

Question

Using python on web2py server I have following problem. How do I loop trough json to find all the keys that have null values (or empty strings) and collect those keys to report what is missing:

This is my json example.

 {
"version" : 1,
"general" : {
    "scriptFrom" : "",
    "scriptTo" : "1.1.2014",
    "scriptComment" : "dada"
},
"nbworkersDays" : [{
        "days" : ["1", "2"],
        "data" : [{
                "nbWorkersFrom" : 2,
                "nbWorkersTo" : null,
                "timeFrom" : "12:00",
                "timeTo" : ""
            }
        ,{
            "nbWorkersFrom" : 2,
            "nbWorkersTo" : 7,
            "timeFrom" : "9:00",
            "timeTo" : "14:00"
        }
    ]
    }
]}

I was thinking to retrieve a list with all keys and if nested than first level . second level. missingData= [scriptFrom, nbworkersDays.nbWorkersTo, nbworkersDays.timeTo]

any suggestions on how to solve this or how would you collect all the errors to report back to client (I have a web app using web2py) thank you

Reut Sharabani · Accepted Answer · 2014-12-18 13:39:20Z

2

You can use a recursive function to iterate over the values of complex objects while remembering the path you're at (to report back):

#!/usr/bin/env python

# used to parse the json text
import json
with open('json.txt', 'r') as f:
    d = json.load(f)
# define list of what your consider as "missing"
missings = [None, ""]

# recursive function to collect missing paths and alter the aggregator list
# make sure to add cases if needed
def aggregate_missing_paths(aggregator, path, item):
    if isinstance(item, list):
        # check all list items
        map(lambda x: aggregate_missing_paths(aggregator, path, x), item)
    if isinstance(item, dict):
        # check all dict items
        map(lambda kv: aggregate_missing_paths(aggregator, path + '.' + kv[0], kv[1]), item.iteritems())
    if item in missings:
        # alter list to cotain path to missing
        aggregator.append(path)

aggregator = []
aggregate_missing_paths(aggregator, 'root', d)
print aggregator

Edit:

I added a version without the aggregator using a recursive generator:

#!/usr/bin/env python

import json
with open('json.txt', 'r') as f:
    d = json.load(f)
missings = [None, ""]
def generate_missing_paths(path, item):
    if isinstance(item, list):
        for v in item:
            for current_path in generate_missing_paths(path, v):
                yield current_path
    if isinstance(item, dict):
        for k, v in item.iteritems():
            for current_path in generate_missing_paths(path + '.' + k, v):
                yield current_path
    if item in missings:
        yield path

for path in generate_missing_paths('root', d):
    print path

edited Dec 18, 2014 at 13:39

answered Dec 18, 2014 at 10:21

Reut Sharabani

31.5k7 gold badges76 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Oscar Over a year ago

Your second version has a problem, I'm not a yield expert, but i think it is causing the problem here as it's keeping the old string pattern and add the new result to it, here is an example of your second versions output: root.nbworkersDays.data.nbWorkersTo root.nbworkersDays.data.nbWorkersTo.timeTo root.nbworkersDays.data.nbWorkersTo.timeTo.general.scriptFrom The first version output the right result: [u'root.nbworkersDays.data.nbWorkersTo', u'root.nbworkersDays.data.timeTo', u'root.general.scriptFrom'] I cannot solve the problem, could you give me a hint here ?

Reut Sharabani Over a year ago

@Oscar- thanks for noticing. It was a simple mixup of variable names. i used path (from the function's parameters) as the loop variable which altered it repeatedly. I now changed to local_path and it should be working.

Collectives™ on Stack Overflow

python parse Json for all null values

1 Answer 1

Edit:

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Edit:

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related