Python: Convert JSON (returned by URL) into List

Question

I am requesting youtube search terms for use with jquery autocomplete, but am having a hard time converting the URL response into a proper format.

In my (Django/Python) view I do:

data2 = urllib2.urlopen('http://suggestqueries.google.com/complete/search?hl=en&ds=yt&client=youtube&hjson=t&jsonp=window.yt.www.suggest.handleResponse&q=jum&cp=3')

(I hardcoded the search term = 'jump' for simplicity)

If I do data2.read() I get what I believe is JSON (copy-pasting the url into a browser also returns this.)

window.yt.www.suggest.handleResponse(["jum",[["jumpstyle","","0"],["jump","","1"],["jump around","","2"],["jump on it","","3"],["jumper","","4"],["jump around house of pain","","5"],["jumper third eye blind","","6"],["jumbafund","","7"],["jump then fall taylor swift","","8"],["jumpstyle music","","9"]],"","","","","",{}])

I need to return this in a format that jquery autocomplete can read. I know it will work if I can get it into a list, for example, mylist = ['jumpstyle', 'jump', 'jump around', ...]

and then convert it back into json before returning it:

json.dumps(mylist)

(This works if I directly define mylist directly as written above.)

But I cannot get from the data that is returned by the URL to either a simple list (which I then convert back into JSON) or to some form of JSON that I can return directly to be used by auto complete.

I've tried, among other things,

j2 = json.loads(data2)

and

j2 = json.loads(data2.read())

Hope someone can help!

Xavier Barbosa · Accepted Answer · 2011-01-08 14:24:22Z

14

remove the &jsonp=window.yt.www.suggest.handleResponse part

import json
import urllib2

data = urllib2.urlopen('http://suggestqueries.google.com/complete/search?hl=en&ds=yt&client=youtube&hjson=t&q=jum&cp=3')

j = json.load(data)
k = [i for i, j, k in j[1]]
l = json.dumps(k)

answered Jan 8, 2011 at 14:24

Xavier Barbosa

3,9471 gold badge22 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Jon Nylander · Accepted Answer · 2011-01-08 14:22:34Z

3

You are doing a JSON-P request which automatically wraps the JSON in a javascript callback function, the one you have specified in the request in fact :)

Strip away the JSON-P parameter from your request and you will get straight JSON directly from the request without having to do any extra python stuff at all.

This should be your request:

http://suggestqueries.google.com/complete/search?hl=en&ds=yt&client=youtube&hjson=t&q=jum&cp=3

and it will return:

["jum",[["jumpstyle","","0"],["jump","","1"],["jump around","","2"],["jump on it","","3"],["jumper","","4"],["jump around house of pain","","5"],["jumper third eye blind","","6"],["jumbafund","","7"],["jump then fall taylor swift","","8"],["jumpstyle music","","9"]],"","","","","",{}]

answered Jan 8, 2011 at 14:22

Jon Nylander

9,0035 gold badges37 silver badges47 bronze badges

2 Comments

dkgirl Over a year ago

Ah... How do I get that to look like: ["jumpstyle", "jump", "jump around"...] I'm not sure how to manipulate these kinds of data: what IS it? A list, a string, a json object? I tried json.loads(returned_by_url) but get an error.

Jon Nylander Over a year ago

Xavier seems to have covered that :)

Mihai Toader · Accepted Answer · 2011-01-08 14:15:04Z

0

The output from the page is not a proper json encoded data. You need to remove the js function call wrapping it.

do this:

import urllib2
import re
import json

data2 = urllib2.urlopen('http://suggestqueries.google.com/complete/search?' +    
   'hl=en&ds=yt&client=youtube&hjson=t&jsonp=window.yt.' + 
   'www.suggest.handleResponse&q=jum&cp=3')

data = re.compile('^[^\(]+\(|\)$').sub('', data2.read())
parsedData = json.loads(data)

parsedData is python array now.

answered Jan 8, 2011 at 14:15

Mihai Toader

12.3k1 gold badge31 silver badges33 bronze badges

Comments

virhilo · Accepted Answer · 2011-01-08 14:21:51Z

0

it's not json it's javascript, if you want to use it as json you must strip the javascript part:

j2 = json.loads(data2[37:-1])

but you can just change the url(remove the 'jsonp=window.yt.www.suggest.handleResponse' part) to have pure json output:

>>> data2 = urllib2.urlopen('http://suggestqueries.google.com/complete/search?hl=en&ds=yt&client=youtube&hjson=t&q=jum&cp=3')
>>> json.loads(data2.read())
[u'jum', [[u'jumpstyle', '', u'0'], [u'jump', '', u'1'], [u'jump around', '', u'2'], [u'jump on it', '', u'3'], [u'jumper', '', u'4'], [u'jump around house of pain', '', u'5'], [u'jumper third eye blind', '', u'6'], [u'jumbafund', '', u'7'], [u'jump then fall taylor swift', '', u'8'], [u'jumpstyle music', '', u'9']], '', '', '', '', '', {}]

edited Jan 8, 2011 at 14:21

answered Jan 8, 2011 at 14:12

virhilo

6,8232 gold badges32 silver badges26 bronze badges

Collectives™ on Stack Overflow

Python: Convert JSON (returned by URL) into List

4 Answers 4

Comments

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related