unable to load json file, huge array of objects, to create a dataframe in python

Question

I'm trying to load a json file which have around 2k objects in an array:

[
  {
    "id": 5375,
    "name": "cepharanthine",
    "mrdef": "The mechanism of action of cepharanthine is multifactorial. The drug exerts membrane effects (modulation of efflux pumps, membrane rigidification) as well as different intracellular and nuclear effects. Cepharanthine interferes with several metabolic axes, primarily with the AMP-activated protein kinase (AMPK) and NFkappaB signaling pathways. In particular, the anti-inflammatory effects of cepharanthine rely on AMPK activation and NFkappaB inhibition.",
    "indications": [
      "Leukopenia",
      " Snake bite - wound",
      " Aptyalism",
      " Alopecia"
    ],
    "contraindication": [
      ""
    ]
  },
  {
    "id": 5301,
    "name": "baloxavir marboxil",
    "mrdef": "Baloxavir marboxil is a prodrug that is converted by hydrolysis to baloxavir, the active form that exerts anti-influenza virus activity. Baloxavir inhibits the endonuclease activity of the polymerase acidic (PA) protein, an influenza virus-specific enzyme in the viral RNA polymerase complex required for viral gene transcription, resulting in inhibition of influenza virus replication. The 50% inhibitory concentration (IC50) of baloxavir was 1.4 to 3.1 nM (n=4) for influenza A viruses and 4.5 to 8.9 nM (n=3) for influenza B viruses in a PA endonuclease assay. Viruses with reduced susceptibility to baloxavir have amino acid substitutions in the PA protein.",
    "indications": [
      "Influenza"
    ],
    "contraindication": [
      ""
    ]
  },
....

I'm trying to load the file like this:

import json

with open('filepath', 'r') as f:
    data = json.load(f)
    print(data)

but the error i'm getting is:

JSONDecodeError                           Traceback (most recent call last)
<ipython-input-81-50fc0a9fd23c> in <module>
      1 with open('C:/Users/Mohammed Safee Uddin/Documents/drugcentral_generics_data.json', 'r') as f:
----> 2     data = json.load(f)
      3     print(data)

~\anaconda3\lib\json\__init__.py in load(fp, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    294         cls=cls, object_hook=object_hook,
    295         parse_float=parse_float, parse_int=parse_int,
--> 296         parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
    297 
    298 

~\anaconda3\lib\json\__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    346             parse_int is None and parse_float is None and
    347             parse_constant is None and object_pairs_hook is None and not kw):
--> 348         return _default_decoder.decode(s)
    349     if cls is None:
    350         cls = JSONDecoder

~\anaconda3\lib\json\decoder.py in decode(self, s, _w)
    335 
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()
    339         if end != len(s):

~\anaconda3\lib\json\decoder.py in raw_decode(self, s, idx)
    353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I tried changing the json.load to json.loads(f.read()) also but nothing works, its throwing same error. I'm new to programming itself, any help will be very much appreciated. Thanks in advance.

Yes I tried but it gives the following error: ``` AttributeError: 'str' object has no attribute 'read' ``` — Mohammed Safee Uddin
– Mohammed Safee Uddin, Commented Jul 23, 2020 at 10:35
stackoverflow.com/a/34010821/8265036 Does this answer you question? — Farhood ET
– Farhood ET, Commented Jul 23, 2020 at 10:38
I assume 'filepath' is replace with the actual file name & path? — Greg
– Greg, Commented Jul 23, 2020 at 10:39
@FarhoodET solution in that link has worked but i'm afraid it has changed the entire data as string, how can I create a dataframe out of it? — Mohammed Safee Uddin
– Mohammed Safee Uddin, Commented Jul 23, 2020 at 10:47

Greg · Accepted Answer · 2020-07-23 11:00:42Z

2

There are 2 (main) ways you can load a json file. Either open the file as a text reader using 'r'. In this case you'll need to call f.read() and convert using json.loads()

with open('data.txt', 'r') as f:
    data = json.loads(f.read())
    print(data)

Alternatively you can open the file, without using reader mode 'r' and call json.load()

with open('data.txt') as json_file:
    data = json.load(json_file)
    print(data)

answered Jul 23, 2020 at 11:00

Greg

4,5383 gold badges19 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Mohammed Safee Uddin Over a year ago

Thanks @Greg, I changed the file to .txt and followed your code, both of them, it worked perfectly. Out of curiosity does this means we cannot upload .json files directly.... :(

Greg Over a year ago

data.txt should have been data.json. As far as I'm aware Python open does not care about the file extention.

Collectives™ on Stack Overflow

unable to load json file, huge array of objects, to create a dataframe in python

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related