Adding a key in an array within a JSON file using Python

Question

I am trying to update the JSON files being updated into my database using the following python script.

#!/usr/bin/env python
# Usage: update json file 
import json
import os

json_dir="Downloads/ADGGTNZ_SERVERFILES/Test_JSON/"
json_dir_processed="Downloads/ADGGTNZ_SERVERFILES/Test_JSON/updated"
for json_file in os.listdir(json_dir):
    if json_file.endswith(".json"):
        processed_json = "%s%s" % (json_dir_processed, json_file)
        json_file = json_dir + json_file
        print "Processing %s -> %s" % (json_file, processed_json)
        with open(json_file, 'r') as f:
            json_data = json.load(f)
            json_data_extract = json_data['grp_cowmonitoring/rpt_animrec'][0]
            if "grp_cowmonitoring/rpt_animrec/grp_animrec/cowtagid" not in json_data_extract:                
                json_data["grp_cowmonitoring/rpt_animrec/grp_animrec/cowtagid"] = json_data["grp_cowmonitoring/rpt_animrec/grp_animrec/damtagid"]
        with open(processed_json, 'w') as f:
            f.write(json.dumps(json_data, indent=4))
    else:
        print "%s not a JSON file" % json_file

The aim of the update script is to find out if "grp_cowmonitoring/rpt_animrec/grp_animrec/cowtagid" lacks within the array on my JSON; Then i will update the same key with a difference on the name "grp_cowmonitoring/rpt_animrec/grp_animrec/damtagid"

original file

{
    "_notes": [], 
    ....
    "grp_cowmonitoring/rpt_animrec": [
        {

            "grp_cowmonitoring/rpt_animrec/grp_animrec/cowtagid": "TZN000403250467", 
            ...
            "grp_cowmonitoring/rpt_animrec/grp_milking/grp_calfreg/rpt_reg_calvedets": [
                {
                    "grp_cowmonitoring/rpt_animrec/grp_milking/grp_calfreg/rpt_reg_calvedets/grp_reg_calvedets/calfsex": "1", 
                    "grp_cowmonitoring/rpt_animrec/grp_milking/grp_calfreg/rpt_reg_calvedets/grp_reg_calvedets/calvtype": "1", 
                    ....
                }
            ], 
            "grp_cowmonitoring/rpt_animrec/anim_weight/weight": "343.0", 
           ...
        }
    ], 
    "fid": 647935, 
    "grp_cowmonitoring/grp-milkuse/milkprocess": "0.0", 
    "start_time": "2018-11-30T08:48:32.278+03", 
    ....
}

Expected JSON file

{
    "_notes": [], 
    ....
    "grp_cowmonitoring/rpt_animrec": [
        {

            "grp_cowmonitoring/rpt_animrec/grp_animrec/cowtagid": "TZN000403250467", 
            ...
            "grp_cowmonitoring/rpt_animrec/grp_milking/grp_calfreg/rpt_reg_calvedets": [
                {
                    "grp_cowmonitoring/rpt_animrec/grp_milking/grp_calfreg/rpt_reg_calvedets/grp_reg_calvedets/calfsex": "1", 
                    "grp_cowmonitoring/rpt_animrec/grp_milking/grp_calfreg/rpt_reg_calvedets/grp_reg_calvedets/calvtype": "1", 
                    "grp_cowmonitoring/rpt_animrec/grp_animrec/damtagid"
                    ....
                }
            ], 
            "grp_cowmonitoring/rpt_animrec/anim_weight/weight": "343.0", 
           ...
        }
    ], 
    "fid": 647935, 
    "grp_cowmonitoring/grp-milkuse/milkprocess": "0.0", 
    "start_time": "2018-11-30T08:48:32.278+03", 
    ....
}

How can i modify my python script to accommodate the changes in my JSON

Error message after update of the original code

Traceback (most recent call last):
  File "/opt/rdm/adggtnz/ADGG-TZA-03/addfidkey2.sh", line 15, in <module>
    json_data_extract = json_data['grp_cowmonitoring/rpt_animrec'][0]
KeyError: 'grp_cowmonitoring/rpt_animrec'

Your code is on the right track, the only problem is how you are accessing the data, you need to do json_data['grp_cowmonitoring/rpt_animrec'][0] to get to the right data. — palvarez
– palvarez, Commented Jul 11, 2019 at 9:29

palvarez · Accepted Answer · 2019-07-11 12:28:47Z

1

You just need to access the right elements from your file:

import json
import os

json_dir="Downloads/ADGGTNZ_SERVERFILES/Test_JSON/"
json_dir_processed="Downloads/ADGGTNZ_SERVERFILES/Test_JSON/updated/"
for json_file in os.listdir(json_dir):
    if json_file.endswith(".json"):
        processed_json = "%s%s" % (json_dir_processed, json_file)
        json_file = json_dir + json_file
        print "Processing %s -> %s" % (json_file, processed_json)
        with open(json_file, 'r') as f:
            json_data = json.load(f)
            json_data_extract = json_data.get('grp_cowmonitoring/rpt_animrec', [])
            for cow in json_data_extract:
                if "grp_cowmonitoring/rpt_animrec/grp_animrec/cowtagid" not in cow:
                    # Skip if cowtagid is not present
                    continue
                calves = cow.get("grp_cowmonitoring/rpt_animrec/grp_milking/grp_calfreg/rpt_reg_calvedets", [])
                for calf in calves:
                    if "grp_cowmonitoring/rpt_animrec/grp_animrec/damtagid" not in calf:
                        print "Updating ..."
                        calf["grp_cowmonitoring/rpt_animrec/grp_animrec/damtagid"] = cow["grp_cowmonitoring/rpt_animrec/grp_animrec/cowtagid"]
        with open(processed_json, 'w') as f:
            f.write(json.dumps(json_data, indent=4))
    else:
        print "%s not a JSON file" % json_file

edited Jul 11, 2019 at 12:28

answered Jul 11, 2019 at 9:34

palvarez

1,6082 gold badges10 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Mirieri Mogaka Over a year ago

i have updated my script but it fails with the error

Mirieri Mogaka Over a year ago

I have copied it at the bottom of my description @palvarez

palvarez Over a year ago

That means not all json files have this key "grp_cowmonitoring/rpt_animrec". You could have a condition to make sure you are modifying the right files maybe. Could you give more details about the files you are treating with?

Mirieri Mogaka Over a year ago

Not all files have this key; all JSON files those with this key "grp_cowmonitoring/rpt_animrec" and those without are under one folder

palvarez Over a year ago

@MirieriMogaka Also it will help to have the initial and expected output file for example

|

Collectives™ on Stack Overflow

Adding a key in an array within a JSON file using Python

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related