Nested data to csv python

Question

I have a file of nested json data. I am trying to "get.some_object" and write a csv file with the objects (I think they are called objects: "some_object": "some_value"); I would like one row for each group of nested items. This is my code:

import csv
import json

path = 'E:/Uni Arbeit/Prof Hayo/Sascha/Bill data/97/bills/hr/hr4242'

outputfile = open('TaxLaw1981.csv', 'w', newline='')                            
outputwriter = csv.writer(outputfile)

with open(path + "/" + "/data.json", "r") as f:
    data = json.load(f)

    for act in data['actions']:
        a = act.get('acted_at')
        b = act.get('text')
        c = act.get('type')

        outputwriter.writerow([a, b, c])
outputfile.close()

The problem I have is that it only writes the last group of data to csv; however when I run

with open(path + "/" + "/data.json", "r") as f:
data = json.load(f)

for act in data['actions']:

    a = act.get('acted_at')
    b = act.get('text')
    c = act.get('type')
    print (a)

all of my "a" values print out.

Suggestions?

ishaan · Accepted Answer · 2016-01-20 10:47:25Z

2

You need to flush your outputwriter to write the row to the file, else it will keep on replacing the one in the variable and eventually only write the last value. Writerow only works when you close the file unless you flush the data.

for act in data['actions']:
        a = act.get('acted_at')
        b = act.get('text')
        c = act.get('type')
        outputwriter.writerow([a, b, c])
        outputfile.flush()

edited Jan 20, 2016 at 10:47

answered Jan 20, 2016 at 10:31

ishaan

2,0511 gold badge20 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Collective Action Over a year ago

I tried your suggestion and got the following error: Traceback (most recent call last): File "E:\Uni Arbeit\Prof Hayo\Sascha\code for parsing senate data .py", line 31, in <module> outputwriter.flush() AttributeError: '_csv.writer' object has no attribute 'flush'

ishaan Over a year ago

So sorry for that, you need to do a outputfile.flush() , editing answer.

ishaan Over a year ago

Yeah, that seems like a json read failure, not sure. Are you sure that file exists at the path or have you changed the permission from 'r' to 'w'?

Collective Action Over a year ago

I changed it from "r" to "wb" and got the error Traceback (most recent call last): File "E:\Uni Arbeit\Prof Hayo\Sascha\code for parsing senate data .py", line 10, in <module> data = json.load(f) File "C:\Python34\lib\json_init_.py", line 265, in load return loads(fp.read(), io.UnsupportedOperation: read

Collective Action Over a year ago

when I use "r" I get a very long error, the one I posted earlier and already deleated.

|

mementum · Accepted Answer · 2016-01-20 14:24:23Z

1

The code you posted above works 100% with the file you have.

The file (for anyone interested) is available with rsync -avz --delete --delete-excluded --exclude **/text-versions/ govtrack.us::govtrackdata/congress/97/bills/hr/hr4242 ..

And the output to the csv file is (omitting some lines in the middle)

1981-07-23,Referred to House Committee on Ways and Means.,referral
1981-07-23,"Consideration and Mark-up Session Held by Committee on Ways and Means Prior to Introduction (Jun 10, 81 through Jul 23, 81).",action
1981-07-23,"Hearings Held by House Committee on Ways and Means Prior to Introduction (Feb 24, 25, Mar 3, 4, 5, 24, 25, 26, 27, 30, 31, Apr 1, 2, 3, 7, 81).",action
...
...
...
1981-08-12,Measure Signed in Senate.,action
1981-08-12,Presented to President.,topresident
1981-08-13,Signed by President.,signed
1981-08-13,Became Public Law No: 97-34.,enacted

You should post the full error code you get when you execute (probably due to an encoding error) to let someone understand why your code is failing.

answered Jan 20, 2016 at 14:24

mementum

3,23315 silver badges20 bronze badges

2 Comments

Collective Action Over a year ago

For some reason, something in my code was erasing the whole file so that when I ran the code twice, there was no code to run it on. I just used my original code now, and it works. This is too bizarre for me to understand

mementum Over a year ago

Swapping r for w as you mentioned in one of your comments may have helped

Collectives™ on Stack Overflow

Nested data to csv python

2 Answers 2

9 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

9 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related