Parsing Twitter json with Python

Question

I'm using twython library in python to dump public tweets of my own. Data are downloaded in json format refer : https://api.twitter.com/1.1/statuses/home_timeline.json

How to print all data line by line, like

    print "Tweet : %s" %tweet['text']#status
    print "Create Time : %s" %tweet['created_at']#time of tweet
    print "Geo location : %s" %tweet['geo']#geo location if avail
    print "Favorite Count : %s" %tweet['favorite_count']
    print "Source : %s" %tweet["source"]
    print "Retweeted : %s" %tweet["retweeted"]
    print "contributors :%s" %tweet["contributors"]
    print "truncated : %s" %tweet["truncated"]
    print "is_quote_status : %s" %tweet["is_quote_status"]
    print "in_reply_to_status_id : %s" %tweet["in_reply_to_status_id"]
    print "Unique ID : %s" %tweet["id"]
    print "coordinates : %s" %tweet["coordinates"]
    print "in_reply_to_screen_name : %s" %tweet["in_reply_to_screen_name"]
    print "retweet_count : %s" %tweet["retweet_count"]
    print "in_reply_to_user_id : %s" %tweet["in_reply_to_user_id"]
    print "favorited :%s" %tweet["favorited"]

If you have the json, you can convert it to python dict using json.loads. — Barun Sharma
– Barun Sharma, Commented Jul 22, 2015 at 9:37
Don't complicate the things, you can use Tweepy module for accessing the tweets. — ZdaR
– ZdaR, Commented Jul 22, 2015 at 9:38
Any sample codes to get started. The format of json output will be available in the link api.twitter.com/1.1/statuses/home_timeline.json — jOSe
– jOSe, Commented Jul 22, 2015 at 9:39
@zdaR tweepy, is it easy as twython? Any pro and con would you like to point out? No much methods available for user timeline tweepy.readthedocs.org/en/v3.2.0/api.html#API.user_timeline — jOSe
– jOSe, Commented Jul 22, 2015 at 9:40
Maybe you could consider using nltk. There are new some functions to read twitter jsons, although not yet released. Please take a look to this notebook, paragraph Extracting Parts of a Tweet. If you wanted to use it, you have to download the project source and add it to your PYTHONPATH. No idea when the actual release will be. (By the way, this is using twython for accessing twitter. Hope it helps. — lrnzcig
– lrnzcig, Commented Jul 23, 2015 at 9:23

Barun Sharma · Accepted Answer · 2015-07-22 10:09:23Z

1

Considering that you have got the tweet in json format using twython and it looks something like:- "{'text' : 'abc', 'created_at': '<created_date>'}"

You can use python json like:-

>>import json
>>tweet_json = <your_json>
>>python_datastruct = json.loads(tweet_json)

Above sample will return you a python data structure whic you can use to print the required info.

EDIT: For a nested object , try something like:-

global_dict = {'a':{'a1':{'a11':1, 'a12':2}, 'a2':3}, 'b':4}
def print_recur(py_item):
    for key, value in py_item.items():
        print key
        if type(value) == dict:
            print_recur(value)
        else:
            print value

print_recur(global_dict)

This would iterate over your nested dictionary to print all keys and values.

edited Jul 22, 2015 at 10:09

answered Jul 22, 2015 at 9:43

Barun Sharma

1,4682 gold badges15 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

jOSe Over a year ago

What happens if the value are nested into another value? How to print the nested value automatically? May sample data {u'contributors': None, u'truncated': False, u'text': u'Dumping data for art. #dataviz', u'is_quote_status': False, u'in_reply_to_status_id': None, u'id': 623497430017839104, u'favorite_count': 0, u'source': u'<a href="twitter.com" rel="nofollow">Twitter Web Client</a>', u'retweeted': False, u'coordinates': None, u'entities': {u'symbols': [], u'user_mentions': [], u'hashtags': [{u'indices': [22, 30], u'text': u'dataviz'}], u'urls': []},continue...

Barun Sharma Over a year ago

You can try printing python_datastruct . This would dump all the data. So you will get the structure of received data. Mostly the structure will be same for all received tweets. And hence you can have a generic function to print your customized data.

jOSe Over a year ago

Generic functions ? Like. Is it similar to this stackoverflow.com/questions/23306653/…

Barun Sharma Over a year ago

Please check the edit....and yes I ment something like what you have pointed. But make it more generic. :)

jOSe Over a year ago

Thanks that was great help, but still one more nested value are stuck.

|

Collectives™ on Stack Overflow

Parsing Twitter json with Python

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related