Remove item in list of dictionaries if value is empty, "null" or None in Python

Question

I have a JSON object which contains all sorts of unnecessary values like "null", "", None. I'd like to remove the whole object if it contains such a value.

>>> json.dumps(event, indent=4)
"event" = {
    "status": "COMPLETED",  
    "dataValues": [
    {
        "key": "wiiDcsQ5pdQ",
        "value": "25"
    },
    {
        "key": "RsZ4gQzWPWU",
        "value": "null"
    },
    {
        "key": "L7aO70bcrbP",
        "value": ""
    },
    {
        "key": "gY6pXRwdThm",
        "value": "null"
    },
    {
        "key": "x1708y0C4C7",
        "value": False
    }
    ]
}

My failed attempts:

no_values = ["null", "", None]

# no changes/deletions:
[elem for elem in event['dataValues'] if elem['value'] not in no_values]

# TypeError: argument of type 'bool' is not iterable
[elem for elem in event['dataValues'] if any(s in elem['value'] for s in no_values]

How can I do so?

Did you try to store your first atempt into an variable? Because it does not modify the original — RvdK
– RvdK, Commented Jan 28, 2016 at 14:49

Anshul Goyal · Accepted Answer · 2016-01-28 14:49:50Z

2

This works for me, where I store the data in a variable and evaluate it using your first list comprehension. Note that I'm using a set of ignored_values, which is slightly faster:

In [12]: import json

In [13]: json_text = """{
    "status": "COMPLETED",  
    "dataValues": [
    {
        "key": "wiiDcsQ5pdQ",
        "value": "25"
    },
    {
        "key": "RsZ4gQzWPWU",
        "value": "null"
    },
    {
        "key": "L7aO70bcrbP",
        "value": ""
    },
    {
        "key": "gY6pXRwdThm",
        "value": null
    },
    {
        "key": "x1708y0C4C7",
        "value": false
    }
    ]
}"""

In [14]: data = json.loads(json_text)

In [15]: ignored_values = set(["null", "", None])

In [16]: [elem for elem in data['dataValues'] if elem['value'] not in ignored_values]
Out[16]: 
[{u'key': u'wiiDcsQ5pdQ', u'value': u'25'},
 {u'key': u'x1708y0C4C7', u'value': False}]

answered Jan 28, 2016 at 14:49

Anshul Goyal

78k42 gold badges161 silver badges196 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ShadowRanger Over a year ago

Minor note: If this is modern Python (Py3.2 or higher), you can get a minor speed up by avoiding the creating of a named ignored_vals set on each use of such a function, instead having Python load a constant frozenset, by doing: [elem for elem in data['dataValues'] if elem['value'] not in {"null", "", None}] The peephole optimizer was upgraded to recognize membership tests in a set literal of Python literals, and converts it to a constant frozenset once at byte code compile time, then reuses it for free thereafter.

Anshul Goyal Over a year ago

@ShadowRanger That's an interesting tidbit, thanks for sharing.

FatmaT · Accepted Answer · 2016-01-28 15:07:08Z

2

I think this is what you want;

ignored_values = ['null', '', None]
data_values = event['dataValues']

In [36]: event['dataValues'] = filter(lambda x: x['value'] not in  ignored_values, data_values) 
In [37]: event
Out[37]:
{'dataValues': [{'key': 'wiiDcsQ5pdQ', 'value': '25'},
  {'key': 'x1708y0C4C7', 'value': False}],
 'status': 'COMPLETED'}

answered Jan 28, 2016 at 15:07

FatmaT

2551 silver badge9 bronze badges

Collectives™ on Stack Overflow

Remove item in list of dictionaries if value is empty, "null" or None in Python

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related