Python JSON parsing not storing values

Question

I've looked over this many times, and cant seem to find the problem with it.

I am trying to pull 3 fields from a JSON response (engagements, shares, comments), sum them together, then print the sum.

It seems to be returning the fields correctly, but it returns zero in the final print.

I'm very new to this stuff, but would appreciate any help anyone can give me. I'm guessing there is something fundamental I am missing here.

import urllib2,time,csv,json,requests,urlparse,pdb


SEARCH_URL = urllib2.unquote("http://soyuz.elastic.tubularlabs.net:9200/intelligence/video/_search?q=channel_youtube_id:""%s""%20AND%20published:%3E20150715T000000Z%20AND%20published:%3C20150716T000000Z")

reader = csv.reader(open('input.csv', 'r+U'), delimiter=',', quoting=csv.QUOTE_NONE)
cookie = {"user": "2|1:0|10:1438908462|4:user|36:eyJhaWQiOiA1Njk3LCAiaWQiOiA2MzQ0fQ==|b5c4b3adbd96e54833bf8656625aedaf715d4905f39373b860c4b4bc98655e9e"}

# idsToProcess = []
# for row in reader:
#   if len(row)>0:
#       idsToProcess.append(row[0])
idsToProcess = ['qdGW_m8Rim4FeMM29keDEg']
for userID in idsToProcess:
#   print "fetching for %s.." % fbid
url = SEARCH_URL % userID
soyuzResponse = None
response = requests.request("GET", url, cookies=cookie)
ret = response.json()
soyuzResponse = ret['hits']['hits'][0]['_source']['youtube']
print soyuzResponse
totalDelta = 0
totalEngagementsVal = 0
totalSharesVal = 0
totalCommentsVal = 0
valuesArr = []
for entry in valuesArr:
        arrEngagements = entry['engagements']
        arrShares = entry['shares'] 
        arrComments = entry['comments'] 

        if len(arrEngagements)>0:
            totalEngagementsVal = arrEngagements
        elif len(arrShares)>0:
            totalSharesVal = arrShares
        elif len(arrComments)>0:
            totalCommentsVal = arrComments

print "%s,%s" % (userID,totalEngagementsVal+totalSharesVal+totalCommentsVal)
totalDelta += totalEngagementsVal+totalSharesVal+totalCommentsVal
time.sleep(0)
print "%s,%s" % (userID,totalDelta)




exit()

Here is the json I am parsing:

   took: 371,
    timed_out: false,
    _shards: {
    total: 128,
    successful: 128,
    failed: 0
    },
    hits: {
    total: 1,
    max_score: 9.335125,
    hits: [
    {
    _index: "intelligence_v2",
    _type: "video",
    _id: "jW7mjVdzR_U",
    _score: 9.335125,
    _source: {
    claim: [
    "Blucollection%2Buser"
    ],
    topics: [
    {
    title_non: "Toy",
    topic_id: "/m/0138tl",
    title: "Toy"
    },
    {
    title_non: "Sofia the First",
    topic_id: "/m/0ncq483",
    title: "Sofia the First"
    }
    ],
    likes: 1045,
    duration: 318,
    channel_owner_type: "influencer",
    category: "Entertainment",
    imported: "20150809T230652Z",
    title: "Princess Sofia Cash Register Toy Surprise - Play Doh Caja Registradora             Disney Sofia the First",
    audience_location: [
    {
    country: "US",
    value: 100
    }
    ],
    comments: 10,
    twitter: {
    tweets: 6,
    engagements: 6
    },
    description: "Disney Princess "Sofia Cash Register" toy unboxing review by          DisneyCollector. This is the authentic Royal toy of Sofia fit for a little     Princess of Enchantia. Young Girls learn early on how mathematics is important in our lives, and learn to do math, developing creativity with a super fun game! Thx 4 watching this "Disney Princess Sofia cash register" unboxing review. In this video i also used Disney Frozen Princess Anna, Nickelodeon Peppa Pig blind bag and plastilina Play-Doh. Revisión del juguete Princesita Sofía Caja Registradora Real para niños y niñas. Las niñas aprenden desde muy temprano cómo las matemáticas es importante en nuestras vidas, y aprenden a hacer matemáticas, el desarrollo de la creatividad con un juego súper divertido! Here's how to say Princess in other languages: printzesa, 公主, prinses, prenses, printsess, princesse, Prinzessin, puteri, banphrionsa, Principesse, principessa, プリンセス, princese, puteri, prinsessa,prinsesse, princesa, công chúa, tywysoges, Princesses Disney, Prinzessinen, 공주, Princesas Disney, Disney πριγκίπισσες, Дисней принцесс, 디즈니 공주, ディズニーのお姫様, Vorstin, koningsdochter, Fürstin, πριγκίπισσα, księżniczka, królewna, принцесса. Here's how register is called in other languages: Caja Registradora de Princesa Sofía, Caisse Enregistreuse Princesse Sofia, Kassa, Registrierkasse Sofia die Erste Auf einmal Prinzessin, Registratore di Cassa di La Principessa Sofia, Caixa Registadora da Princesa Sofia, ηλεκτρονική ταμειακή μηχανή Σοφία η Πριγκίπισσα, 電子式金銭登録機 ちいさなプリンセス ソフィア, София Прекрасная кассовый аппарат, 디즈니주니어 리틀 프린세스 소피아 전자 금전 등록기, máy tính tiền điện tử, daftar uang elektronik, elektronik yazarkasa, Sofia den första kassaapparat leksak, Jej Wysokość Zosia kasa zabawki, Sofia het prinsesje kassa speelgoed, София Първа касов апарат играчка, casa de marcat jucărie Sofia Întâi. Princess Sofia SLEEPOVER Slumber Party - Princesita Sofía Pijamada Real. https://www.youtube.com/watch?v=WSa-Tp7HfyQ Princesita Sofía Castillo Mágico Parlante juguete de niñas. https://www.youtube.com/watch?v=ALQm_3uhIyg Sofia the First Magical Talking Castle Royal Prep Academy. https://www.youtube.com/watch?v=gcUiY0Suzrc Play-Doh Meal Makin' Kitchen with Princess Sofia the First. https://www.youtube.com/watch?v=x_-OxnRXj6g Sofia the First Royal Prep Academy Dolls Character Collection. https://www.youtube.com/watch?v=_kNY6AkSp9g Peppa Pig Picnic Adventure Car With Princess Sofia the First. https://www.youtube.com/watch?v=KIPH3txlq1o Watch "Sofia the First Talking Magic Castle" talking Clover: https://www.youtube.com/watch?v=ALQm_3uhIyg Play-Doh Sofia the First Magic Talking Castle w/ Peppa Pig: https://www.youtube.com/watch?v=-slXqMiDrY0 Play-Doh Sofia the First Going to School Portable Classroom http://www.youtube.com/watch?v=0R-dkVAIUlA",
views: 941726,
channel_network: null,
channel_subscribers: 5054024,
youtube_id: "jW7mjVdzR_U",
facebook: {
engagements: 9,
likes: 2,
shares: 7,
comments: 0
},
location_demo_count: 1,
is_public: true,
engagements: 1070,
channel_country: "US",
demo_count: null,
monetizable: true,
youtube: {
engagements: 1055,
likes: 1045,
comments: 10
},
published: "20150715T100003Z",
channel_youtube_id: "qdGW_m8Rim4FeMM29keDEg"
}
}
]
}
}

Response from terminal after running script:

{u'engagements': 1055, u'likes': 1045, u'comments': 10}
qdGW_m8Rim4FeMM29keDEg,0
qdGW_m8Rim4FeMM29keDEg,0

Cyphase · Accepted Answer · 2015-08-10 00:21:02Z

1

Your problem is these two lines:

valuesArr = []
for entry in valuesArr:

Because valuesArr is empty, the for loop never iterates, and that's where your totals are being summed.

answered Aug 10, 2015 at 0:21

Cyphase

12k3 gold badges33 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Waveformer Over a year ago

Thanks. I tried adding valuesArr = ret['hits']['hits'][0]['_source']['youtube'] but it returned "TypeError: string indices must be integers".

Cyphase Over a year ago

While this code definitely has a smell, in this case you'd want valuesArr = [ret['hits']['hits'][0]['_source']['youtube']]. Note that ret['hits']['hits'][0]['_source']['youtube'] is inside a one-element list.

Cyphase Over a year ago

@Waveformer, glad I could help :). In that case, can you accept it so that the question will be marked as answered?

Waveformer Over a year ago

There does not appear to be any way to mark your comment as an answer .

Cyphase Over a year ago

@Waveformer, I mean accept my answer, not my comment. As the questioner, you can mark an answer as 'accepted'. There should be an outline of a checkmark to the left of the answer, under the vote total.

Collectives™ on Stack Overflow

Python JSON parsing not storing values

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related