Convert Nested JSON to Excel using Python

Question

I want to convert Nested JSON to Excel file format using Python. I've done nearly as per requirements but I want to achieve excel format as below.

JSON

[
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Cooktops/zgbs/appliances/3741261",
    "subCategory": [
      
    ],
    "title": "Cooktops"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Dishwashers/zgbs/appliances/3741271",
    "subCategory": [
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Built-Dishwashers/zgbs/appliances/3741281",
        "subCategory": [
          
        ],
        "title": "Built-In Dishwashers"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Portable-Countertop-Dishwashers/zgbs/appliances/3741301",
        "subCategory": [
          
        ],
        "title": "Portable & Countertop Dishwashers"
      }
    ],
    "title": "Dishwashers"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Freezers/zgbs/appliances/3741331",
    "subCategory": [
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Chest-Freezers/zgbs/appliances/3741341",
        "subCategory": [
          
        ],
        "title": "Chest Freezers"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Upright-Freezers/zgbs/appliances/3741351",
        "subCategory": [
          
        ],
        "title": "Upright Freezers"
      }
    ],
    "title": "Freezers"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Ice-Makers/zgbs/appliances/2399939011",
    "subCategory": [
      
    ],
    "title": "Ice Makers"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Range-Hoods/zgbs/appliances/3741441",
    "subCategory": [
      
    ],
    "title": "Range Hoods"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Ranges/zgbs/appliances/3741411",
    "subCategory": [
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Drop-Ranges/zgbs/appliances/3741421",
        "subCategory": [
          
        ],
        "title": "Drop-In Ranges"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Freestanding-Ranges/zgbs/appliances/3741431",
        "subCategory": [
          
        ],
        "title": "Freestanding Ranges"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Slide-Ranges/zgbs/appliances/2399946011",
        "subCategory": [
          
        ],
        "title": "Slide-In Ranges"
      }
    ],
    "title": "Ranges"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Refrigerators/zgbs/appliances/3741361",
    "subCategory": [
      
    ],
    "title": "Refrigerators"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Wall-Ovens/zgbs/appliances/3741481",
    "subCategory": [
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Combination-Microwave-Wall-Ovens/zgbs/appliances/3741491",
        "subCategory": [
          
        ],
        "title": "Combination Microwave & Wall Ovens"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Double-Wall-Ovens/zgbs/appliances/3741501",
        "subCategory": [
          
        ],
        "title": "Double Wall Ovens"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Single-Wall-Ovens/zgbs/appliances/3741511",
        "subCategory": [
          
        ],
        "title": "Single Wall Ovens"
      }
    ],
    "title": "Wall Ovens"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Warming-Drawers/zgbs/appliances/2399955011",
    "subCategory": [
      
    ],
    "title": "Warming Drawers"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Washers-Dryers/zgbs/appliances/2383576011",
    "subCategory": [
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Clothes-Dryers/zgbs/appliances/13397481",
        "subCategory": [
          
        ],
        "title": "Dryers"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Clothes-Washing-Machines/zgbs/appliances/13397491",
        "subCategory": [
          
        ],
        "title": "Washers"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Combination-Washers-Dryers/zgbs/appliances/13755271",
        "subCategory": [
          
        ],
        "title": "All-in-One Combination Washers & Dryers"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Stacked-Washer-Dryer-Units/zgbs/appliances/2399957011",
        "subCategory": [
          
        ],
        "title": "Stacked Washer & Dryer Units"
      }
    ],
    "title": "Washers & Dryers"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Wine-Cellars/zgbs/appliances/3741521",
    "subCategory": [
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Built-Wine-Cellars/zgbs/appliances/3741551",
        "subCategory": [
          
        ],
        "title": "Built-In Wine Cellars"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Freestanding-Wine-Cellars/zgbs/appliances/3741541",
        "subCategory": [
          
        ],
        "title": "Freestanding Wine Cellars"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Furniture-Style-Wine-Cellars/zgbs/appliances/3741561",
        "subCategory": [
          
        ],
        "title": "Furniture-Style Wine Cellars"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Small-Wine-Cellars/zgbs/appliances/3741531",
        "subCategory": [
          
        ],
        "title": "Small Wine Cellars"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Wine-Cellar-Cooling-Systems/zgbs/appliances/3741581",
        "subCategory": [
          
        ],
        "title": "Wine Cellar Cooling Systems"
      },
      {
        "url": "https://www.amazon.com/Best-Sellers-Appliances-Wine-Rooms/zgbs/appliances/3741571",
        "subCategory": [
          
        ],
        "title": "Wine Rooms"
      }
    ],
    "title": "Wine Cellars"
  },
  {
    "url": "https://www.amazon.com/Best-Sellers-Appliances-Home-Appliance-Warranties/zgbs/appliances/2242350011",
    "subCategory": [
      
    ],
    "title": "Appliance Warranties"
  }
]

I'm traversing all subCategories like this:

row = 1

def TraverseJSONTree(jsonObject, count=0):
    title = jsonObject.get('title')
    url = jsonObject.get('url')

    print 'Title: ' + title + ' , Position: ' + str(count)

    worksheet.write_string(row, count, title)
    worksheet.write_string(row, 6, url)
    global row
    row+=1 

    subCategories =  jsonObject.get('subCategory',[])

    for category in subCategories:
        TraverseJSONTree(category, count+1)


for jsonObject in json.loads(jsonArray):
    TraverseJSONTree(jsonObject)

Expected Result

Can you tell me how to? I've managed to do nearly as per requirement. I'm newbie in Python — Asad Haider
– Asad Haider, Commented Sep 9, 2016 at 6:25

d-ashu · Accepted Answer · 2016-09-09 08:53:34Z

2

row = 1

def TraverseJSONTree(jsonObject, main_title=None, count=0):
    if main_title is None:
        main_title = title = jsonObject.get('title')
    else:
        title = jsonObject.get('title')
    url = jsonObject.get('url')

    print 'Title: ' + title + ' , Position: ' + str(count)

    if main_title is not None:
        worksheet.write_string(row, 0, title)
    worksheet.write_string(row, count, title)
    worksheet.write_string(row, 6, url)
    global row
    row+=1 

    subCategories =  jsonObject.get('subCategory',[])

    for category in subCategories:
        TraverseJSONTree(category, main_title, count+1)

for jsonObject in json.loads(jsonArray):
    TraverseJSONTree(jsonObject)

it will return your expected output as it needs a check if category is there then you have to right the original title on the 0th col in excel reamin as same.

answered Sep 9, 2016 at 8:53

d-ashu

576 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Asad Haider Over a year ago

It's not giving the desired output as mentioned above. I've checked your script and it's give me this: docs.google.com/spreadsheets/d/…

Asad Haider Over a year ago

All i need should be like this: docs.google.com/spreadsheets/d/… and it should work if there are more sub categories of sub categories.

d-ashu Over a year ago

Ahh silly mistake just update title with main_title in following line :if main_title is not None: worksheet.write_string(row, 0, main_title)

Asad Haider Over a year ago

Thanks, Its working fine if there is only one subcategory. How can i make it worked if there are multiple sub to sub categories?

Neeraj Komuravalli · Accepted Answer · 2016-09-09 09:46:09Z

2

Modification : Simplest way to do this would be to use csv module, say we have the whole json in the variable a

import csv
import cPickle as pickle 

fieldnames = ['Category1', 'Category1.1', 'url']
csvfile = open("category.csv", 'wb')
csvfilewriter = csv.DictWriter(csvfile, fieldnames=fieldnames,dialect='excel', delimiter=',')
csvfilewriter.writeheader()

for b in a:     
    data = []
    data.append(b['title'])
    data.append("")
    data.append(b['url'])
    csvfilewriter.writerow(dict(zip(fieldnames,data)))
    data = []
    for i in xrange(len(b['subCategory'])):
        data.append(b['title'])
        data.append(b['subCategory'][i]['title'])
        data.append(b['subCategory'][i]['url'])
        csvfilewriter.writerow(dict(zip(fieldnames,data)))

You will have the desired csv in the same location. This works for only two subcategories (because i have checked the data given by you and say there were only two categories (ie 1 and 1.1)) but in case you want for more than repeat the same(I know it's not the most efficient way couldn't think of any in such a short time)

You can also use pandas module to convert the dictionary import pandas as pd pd.DataFrame.from_dict(dcitionaty_element)

And then do it on all the dictionaries in that json and merge them and save it to a csv file.

edited Sep 9, 2016 at 9:46

answered Sep 9, 2016 at 7:49

Neeraj Komuravalli

3062 silver badges8 bronze badges

3 Comments

Asad Haider Over a year ago

can this would work if there are subCategories of subCategories?

Neeraj Komuravalli Over a year ago

if you know what are max no of sub categories, the i can do that just need to use the for loop again.

Saeed Over a year ago

Python 3 hint: csvfile = open("category.csv", 'w')

Collectives™ on Stack Overflow

Convert Nested JSON to Excel using Python

2 Answers 2

4 Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related