which is the most efficient way of loading a JSON dataset into Pandas DataFrames

Question

I didn't find anything in the Pandas documentations and cookbook (just references to CSV, and text files with separators) on JSON.

Is there an already defined function to load JSON directly into DataFrame? If there are different alternatives, which is the most efficient?

Is not a particular instance question. I would like to know something that works for different structures. In a particular case, I could write an specific parser, iterating it, and pushing data in to the DF. — Luchux
– Luchux, Commented May 8, 2013 at 15:58
I was sure there was some discussion of a from_json method in the past, can't put my finger on it. Similar to this issue... — Andy Hayden
– Andy Hayden, Commented May 8, 2013 at 16:02

erantdo · Accepted Answer · 2013-08-08 07:51:30Z

11

with pandas 0.12:

import pandas as pd

d = pd.read_json('JSON File')

answered Aug 8, 2013 at 7:51

erantdo

7052 gold badges10 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

arpiagar · Accepted Answer · 2014-05-25 08:35:41Z

9

The generic way to laod JSON to DataFrame is mentioned above:

import pandas as pd
d = pd.read_json('JSON File')

However, if your JSON file is nested and you need to create DataFrame of some nested attribute in it, one can use

 from pandas.io.json import json_normalize
 json_normalize(df[JSONKEYWORD])

In the JSONKEYWORD one can pass the nested JSON object and you get a sub Data Frame for that Nested JSON object.

answered May 25, 2014 at 8:35

arpiagar

4,7621 gold badge21 silver badges12 bronze badges

Comments

James · Accepted Answer · 2013-05-13 13:59:43Z

0

Install pandasjson from github which provides DataFrame from_json and to_json classmethods.

https://github.com/pydata/pandasjson

import pandasjson
from pandas import DataFrame

"""
pinfo DataFrame.from_json
File:  ../lib/python2.7/site-packages/pandasjson.py
Definition: DataFrame.from_json(cls, json, orient='columns', dtype=None, numpy=True)

pinfo DataFrame.to_json
File:  ../lib/python2.7/site-packages/pandasjson.py
Definition: DataFrame.to_json(self, orient='columns', double_precision=10, force_ascii=True)
"""

answered May 13, 2013 at 13:59

James

91 bronze badge

Collectives™ on Stack Overflow

which is the most efficient way of loading a JSON dataset into Pandas DataFrames

3 Answers 3

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related