How to merge multiple json files into one file in python

Question

I want to merge multiple json files into one file in python. The thing that I want to do is if there are several .json files like:

# temp1.json
[{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'}]

# temp2.json
[{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'}]

# temp3.json
[{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]

The result.json files I want to get should look like:

# result.json
[{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'},
{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'},
{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]

The result.json files I got is:

# result.json
[[{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'}],
[{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'}],
[{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]]

I used the code to merge .json files from here and changed it very slightly like below:

files=['my.json','files.json',...,'name.json']

def merge_JsonFiles(filename):
    result = list()
    for f1 in filename:
        with open(f1, 'r') as infile:
            result.append(json.load(infile))

    with open('counseling3.json', 'w') as output_file:
        json.dump(result, output_file)

merge_JsonFiles(files)

I already read several related questions, but there is no answer I need. Can anyone help me?

use extend instead of append.

Paul Rooney
– Paul Rooney

2019-08-09 02:18:35 +00:00
Commented Aug 9, 2019 at 2:18 — Paul Rooney
– Paul Rooney, Commented Aug 9, 2019 at 2:18
@PaulRooney thanks:)

wendykr
– wendykr

2019-08-09 02:33:34 +00:00
Commented Aug 9, 2019 at 2:33 — wendykr
– wendykr, Commented Aug 9, 2019 at 2:33
function names should be lowercase

Michael
– Michael

2022-12-01 13:47:34 +00:00
Commented Dec 1, 2022 at 13:47 — Michael
– Michael, Commented Dec 1, 2022 at 13:47

Akaisteph7 · Accepted Answer · 2019-08-09 02:18:56Z

21

You should use extend instead of append. It will add the items of the passed list to result instead of a new list:

files=['my.json','files.json',...,'name.json']

def merge_JsonFiles(filename):
    result = list()
    for f1 in filename:
        with open(f1, 'r') as infile:
            result.extend(json.load(infile))

    with open('counseling3.json', 'w') as output_file:
        json.dump(result, output_file)

merge_JsonFiles(files)

answered Aug 9, 2019 at 2:18

Akaisteph7

6,9222 gold badges39 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

The_Nabzter Over a year ago

This only returns a list of key elements in the json file without the values.

Gryu · Accepted Answer · 2020-08-06 09:25:34Z

7

import json
import pandas as pd

with open('example1.json') as f1:               # open the file
    data1 = json.load(f1)

with open('example2.json') as f2:                # open the file       
    data2 = json.load(f2)
    
df1 = pd.DataFrame([data1])                      # Creating DataFrames
df2 = pd.DataFrame([data2])                      # Creating DataFrames

MergeJson = pd.concat([df1, df2], axis=1)         # Concat DataFrames

MergeJson.to_json("MergeJsonDemo.json")          # Writing Json

edited Aug 6, 2020 at 9:25

Gryu

2,2272 gold badges19 silver badges32 bronze badges

answered Aug 6, 2020 at 8:34

Anjali Bhagat

711 silver badge1 bronze badge

2 Comments

Ryan Hart Over a year ago

I love how clean this looks from a code readability standpoint.

Dmitrij Holkin Over a year ago

Arguments: ('ValueError', ValueError("DataFrame columns must be unique for orient='columns'."))

Abhi · Accepted Answer · 2022-04-23 09:11:52Z

7

files=['my.json','files.json',...,'name.json']

with open('merged_file_name.json', "w") as outfile:
   outfile.write('{}'.format('\n'.join([open(f, "r").read() for f in files])))

answered Apr 23, 2022 at 9:11

Abhi

711 silver badge2 bronze badges

1 Comment

keiv.fly Over a year ago

When answering a question it is better to write some description in addition to code.

YScharf · Accepted Answer · 2022-06-02 20:21:11Z

1

If the goal is to merge several JSON files, try the following:

def combine_jsons():
    file_list = ['first.json', 'second.json',... ,'last.json']
    all_data_dict = {}
    for json_file in file_list:
       with open(json_file,'r+') as file:
           # First we load existing data into a dict.
           file_data = json.load(file)
       all_data_dict.update(file_data)
    with open('merged_data.json', "w") as outfile:# save to json file
        json.dump(all_data_dict, outfile)

If you it must be a list for some reason just change the last line

json.dump([all_data_dict], outfile)

Based on How do I merge two dictionaries in a single expression (take union of dictionaries)?

edited Jun 2, 2022 at 20:21

answered Feb 28, 2022 at 15:52

YScharf

2,16022 silver badges27 bronze badges

2 Comments

lord stock Over a year ago

this givers error

YScharf Over a year ago

What error? Indentation? fixed!

Talex Maxim · Accepted Answer · 2023-04-19 14:54:42Z

0

def merge_files(self):
    res = list()
    for f1 in self.listDomainToMerge:
        item = json.load(open(f1, encoding="utf-8"))
        res.append(item)

    with open('../Configuration%s/configuration.json' % self.owner, 'w') as output:
        output.write(json.dumps(res, ensure_ascii=False, indent=2))

edited Apr 19, 2023 at 14:54

answered Apr 19, 2023 at 14:53

Talex Maxim

11 bronze badge

Comments

Kay Zhang · Accepted Answer · 2019-08-09 03:10:10Z

There is another way.Just loads the json text in those files as a python list, and add them together. Code as below.

# temp1.json
json_a = [{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'}]

# temp2.json
json_b = [{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'}]

# temp3.json
json_c = [{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]

print(json_a + json_b + json_c)

Output:

[{'num': '1', 'item': 'smartphone', 'data': '2019-01-01'},
 {'num': '2', 'item': 'smartphone', 'data': '2019-01-02'},
 {'num': '3', 'item': 'smartphone', 'data': '2019-01-03'},
 {'num': '4', 'item': 'smartphone', 'data': '2019-01-04'},
 {'num': '5', 'item': 'smartphone', 'data': '2019-01-05'},
 {'num': '6', 'item': 'smartphone', 'data': '2019-01-06'},
 {'num': '7', 'item': 'smartphone', 'data': '2019-01-07'},
 {'num': '8', 'item': 'smartphone', 'data': '2019-01-08'},
 {'num': '9', 'item': 'smartphone', 'data': '2019-01-09'},
 {'num': '10', 'item': 'smartphone', 'data': '2019-01-10'},
 {'num': '11', 'item': 'smartphone', 'data': '2019-01-11'},
 {'num': '12', 'item': 'smartphone', 'data': '2019-01-12'}]

Collectives™ on Stack Overflow

How to merge multiple json files into one file in python

6 Answers 6

1 Comment

2 Comments

1 Comment

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

1 Comment

2 Comments

1 Comment

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related