KeyError In Python With json.dumps

Question

I'm trying to work with dictionaries inside a list in a JSON file. The data imports fine and reads fine. For the life of me I can't figure out how to printout the "member_id" keys. I just want to print the list of "member_id" numbers. I was initially using json.loads, then switched to json.dumps. Any help would really be appreciated.

import urllib2
import json
nyt_api_key = '72c9a68bbc504e91a3919efda17ae621%3A7%3A70586819'
url= 'http://api.nytimes.com/svc/politics/v3/us/legislative/congress/113'
json_obj = urllib2.urlopen(url)
data = json.load(json_obj)
data2 = json.dumps(data, sort_keys=True, indent=True, skipkeys = True)

print data2

Output from print data2: (The list goes on and on so it is truncated. There is a closing bracket at the bottom of the list. So it's dictionaries within a list.)

 "positions": [
 {
  "dw_nominate": "0.466", 
  "member_id": "A000055", 
  "vote_position": "Yes"
 }, 
 {
  "dw_nominate": "0.995", 
  "member_id": "A000367", 
  "vote_position": "Yes"
 }, 
 {
  "dw_nominate": "0.666", 
  "member_id": "A000369", 
  "vote_position": "Yes"
 },

Output from print data2['member_id'], output is the same if using 'positions', 'vote_position', etc.:

Traceback (most recent call last):
  File "/Users/Owner/PycharmProjects/untitled2/1", line 9, in <module>
    print data2["positions"]
TypeError: string indices must be integers, not str

Output from print data:

u'positions': [{u'dw_nominate': u'0.466', u'vote_position': u'Yes', u'member_id': u'A000055'}, {u'dw_nominate': u'0.995', u'vote_position': u'Yes', u'member_id': u'A000367'}, {u'dw_nominate': u'0.666', u'vote_position': u'Yes', u'member_id': u'A000369'}

Output from print data['positions']:

print data["positions"] KeyError: 'positions'

Output from print.data(keys):

[u'status', u'results', u'copyright']

Process finished with exit code 0

json.dumps returns string and hence the exception. try print data['member_id'] — Akash Shende
– Akash Shende, Commented Dec 29, 2014 at 18:35
json.dumps produces a string so that's what you're assigning to data2 (from object data which is apparently a list of dicts). What are you trying to accomplish?! — Alex Martelli
– Alex Martelli, Commented Dec 29, 2014 at 18:36
@AkashShende it looks like data is a list of dicts so data['member_id'] will also fail -- something like data[0]['member_id'] might be fine, but with no idea of what the OP is trying to accomplish it's essentially just a guess:-) — Alex Martelli
– Alex Martelli, Commented Dec 29, 2014 at 18:37
@AlexMartelli, I just want to print the list of "member_id" numbers. — Padraic Cunningham
– Padraic Cunningham, Commented Dec 29, 2014 at 18:42
Output from print data: u'positions': [{u'dw_nominate': u'0.466', u'vote_position': u'Yes', u'member_id': u'A000055'}, {u'dw_nominate': u'0.995', u'vote_position': u'Yes', u'member_id': u'A000367'}, {u'dw_nominate': u'0.666', u'vote_position': u'Yes', u'member_id': u'A000369'} Using print data["member_id"] or print data ["u'member_id"] I still get a KeyError. The for loop mentioned below by @PadraicCunningham returns TypeError: string indices must be integers, not str. I haven't exhausted that solution yet, still trying a couple of things with it. @AlexMartelli @Akash Shende — Max Phillips
– Max Phillips, Commented Dec 29, 2014 at 22:36

Padraic Cunningham · Accepted Answer · 2014-12-29 23:17:22Z

1

I just want to print the list of "member_id" numbers.

So you need to loop over positions and access the member_id in each dict:

data ={"positions": [
 {
  "dw_nominate": "0.466",
  "member_id": "A000055",
  "vote_position": "Yes"
 },
 {
  "dw_nominate": "0.995",
  "member_id": "A000367",
  "vote_position": "Yes"
 },
 {
  "dw_nominate": "0.666",
  "member_id": "A000369",
  "vote_position": "Yes"
 }]}

print([d["member_id"] for d in data["results"]["positions"]])
['A000055', 'A000367', 'A000369']

If you look at the API documentation there are examples of each json response.

edited Dec 29, 2014 at 23:17

answered Dec 29, 2014 at 18:39

Padraic Cunningham

181k30 gold badges264 silver badges327 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Max Phillips Over a year ago

as soon as I get it to work I'll post the answer or the work around

Pablo Díaz Ogni · Accepted Answer · 2014-12-30 13:28:09Z

1

data2 is a string value, it doesn't have keys. I think what you want to print is data["positions"]

That's a weird output from data, you don't even have the braces. Try printing the type(data), it should be dict

edited Dec 30, 2014 at 13:28

answered Dec 29, 2014 at 18:33

Pablo Díaz Ogni

1,0438 silver badges12 bronze badges

Comments

Max Phillips · Accepted Answer · 2014-12-30 03:35:40Z

So I should change the heading of this to Scrapping JSON for XML in Python. I'm sure not everyone else would have the same issues I did with JSON but after many frustrating hours I decided to go down path #2... the xml version. The xml version was much easier to work with right out of the gate. In about 1/10 the time I got what I was looking for.

from urllib2 import urlopen
from xml.dom import minidom

feed = urlopen("http://api.nytimes.com/svc/politics/v3/us/legislative.xml?

doc = minidom.parse(feed)
id_element = doc.getElementsByTagName("member_id")
id_number0 = id_element[0].childNodes[0].nodeValue #just a sample
id_number1 = id_element[1].childNodes[0].nodeValue #just a sample
id_number2 = id_element[2].childNodes[0].nodeValue #just a sample

print len(id_element) #to see how many items were in the variable 


count = 0
for item in id_element:
    print id_element[count].childNodes[0].nodeValue
    count = count + 1
    if count == 434:
    break

This is definitely not the cleanest loop. I'm still working on that. But the code solves the problem that I had originally posted. The API key is not the actual one, formatting in the answer window was throwing it off so I just erased a bunch of it. You can find the API at the NYT developer website.

Thanks to everyone who posted.

Collectives™ on Stack Overflow

KeyError In Python With json.dumps

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related