How to handle missing keys in list of JSON objects in Python

Question

I have a list of dicts that are JSON objects, like this:

[{'addressId': 12345, 'city': 'London', 'country': 'UK'}, {'addressId': 67890, 'city': 'Berlin'}

For some of the entries there are some JSON keys missing (not just the values) - in the example above the second address is missing the country. This causes problems down the line when I want to write this data to MySQL.

So I think what I want to do is add the missing keys with empty values. I'm wondering (a) if this is the right approach, and (b) how I would go about doing that.

FWIW, we're talking about a few million rows.

Any hints are greatly appreciated.

Tob · Accepted Answer · 2020-11-05 20:01:42Z

3

Whether or not this is the correct approach depends on the use case down the line. You could simply loop through each element in the list, and, if the key doesn't exist, add it with a default value:

for obj in arr:
    if key1 not in obj:
       obj[key1] = # default value
    # and continue for other keys

answered Nov 5, 2020 at 20:01

Tob

1,0352 gold badges11 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Sebastian Over a year ago

Hmm, I'm trying this: for item in resp: if country not in item: item[country] = "missing" but I get "NameError: name 'country' is not defined". Any ideas?

Tob Over a year ago

country should be in quotes like so: if "country" not in item: item["country"] = "missing"

Lauren Boland · Accepted Answer · 2020-11-05 20:24:35Z

2

I would recommend just doing dictionary.get() for each key at the point of putting the info into the database, that way you can add a default value there if it's missing.

objects = [{'addressId': 12345, 'city': 'London', 'country': 'UK'}, {'addressId': 67890, 'city': 'Berlin'}]

for object in objects:
   country = object.get('country', None)
   city = object.get('city', None)

The default return value for .get() if it doesn't find the key, is None. But I included it in there to show you can put anything there.

answered Nov 5, 2020 at 20:24

Lauren Boland

1718 bronze badges

1 Comment

Lauren Boland Over a year ago

I would recommend this over adding a default value to your JSON, because it will be a memory saver.

user12925425 · Accepted Answer · 2020-11-05 20:28:47Z

-1

If data is not uniform or difficult to present in tabular format than mongodb is the better option. if strict to MYSQL than As Tob suggested ,make the default values .

answered Nov 5, 2020 at 20:28

user12925425

244 bronze badges

Collectives™ on Stack Overflow

How to handle missing keys in list of JSON objects in Python

3 Answers 3

2 Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related