loop through multiple subdirectories, return specific .json file and add content of json file to pandas dataframe

Question

I have a folder with multiple subdirectories that contain a number of .json files.

I am only interested in getting the content of .json files that are titled "all_content.json" in each subdirectory (this file has the same name in each directory).

Then I want to take the content from each file and add it to one pandas dataframe, where the column title is the key (e.g.: column 1 = content and column 2 = date)

{ "content": "The flowers are so pretty here", "date": "1999-10-22" }

This is what I have tried so far, but I am not sure how to select the right file, open it and then save the content:

path = './folders'

for root, dirs, files in os.walk(path):
    print(files) # returns list of all files in the folder 
           for file in files:
    if file.endswith("all_content.json"):
        print(file)
        with open(file) as fp:
            data = json.load(fp)

msa · Accepted Answer · 2023-08-16 21:16:33Z

0

I found an answer myself, but I am not sure if this is the most efficient way. Any suggestions for improvements are appreciated.

files = Path(path).glob("**/all_content.json")

for file in files:
    with open(file) as datafile:
        data = json.load(datafile) # dict
        print(data)
        d = data.values() # dict_values
        content = list(d)[0]
        print(content)
        date = list(d)[1] 
        print(date)

answered Aug 16, 2023 at 21:16

msa

73510 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

loop through multiple subdirectories, return specific .json file and add content of json file to pandas dataframe

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related