How to load data from multiple JSON files from a list using python?

Question

Essentially, I have a list of JSON files that I need information extracted from which is saved in a variable called json_files.

The information from these files needs to then be aggregated and transferred into a new file called summary.json.

Currently, I am able to do this with the following lines of code but only for one file at a time by specifying an index:

with open(json_files[1]) as f:
    data = json.load(f)

with open('summary.json', 'w') as f:
    json.dump(data, f, indent=2)

However, if I try using a for loop like as follows, I get a TypeError: list indices must be integers or slicers, not str

for i in json_files:
    with open(json_files[i]) as f:
        data = json.load(f)

with open('summary.json', 'w') as f:
    json.dump(data, f, indent=2)

Thus, I was wondering what would be the best way to load information from multiple json files and then combine that into 1 new large json file?

Thanks in advance

Does this answer your question? TypeError: list indices must be integers, not str Python — Chris
– Chris, Commented Feb 22, 2020 at 20:54
Try to use the range() function to iterate over the list, by setting range to the length of list. For error : You get the error because for loop is returning the element of list, not index. — Syed Muhammad Arslan
– Syed Muhammad Arslan, Commented Feb 22, 2020 at 20:55

Audemed · Accepted Answer · 2020-02-22 21:25:47Z

3

for i in json_files:
    with open(json_files[i]) as f:
        data = json.load(f)

Should be

for i in json_files:
    with open(i) as f:
        data = json.load(f)

What's happening: json_files is a list with names I'm assuming, so something like json_files = ['file1.json', 'file2.json']

So your open method was getting parameters like json_files['file1.json'] within the for loop which doesn't make sense since json_files is a list and not a dictionary.

The i in the for loop holds the actual value in the list, not the indexes.

Edit:

The data variable in the above code is being overwritten in each iteration of the loop! You probably need to do something like this to fix it:

combined = []
    for i in json_files:
        with open(i) as f:
            combined.append(json.load(f))

And then use the dump function to write this list into another file.

edited Feb 22, 2020 at 21:25

answered Feb 22, 2020 at 20:54

Audemed

2811 silver badge5 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

awdoibbbbbbb2 Over a year ago

Thanks for this. However, this change only loads and dumps the information in the last file of the list. How can I copy and transfer the information of every file in the list to the same new file?

Audemed Over a year ago

Ah, that's because the data variable is being overwritten by each iteration of the loop, so only the last value is held in that one when its used as a parameter for the dump function. I've edited my answer to give you some idea of how I would fix this.

Collectives™ on Stack Overflow

How to load data from multiple JSON files from a list using python?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related