Python read json from url

Question

With Python 3, I want to read a JSON from url. This is what I got:

import json
import urllib.request
url_address = 'https://api.twitter.com/1.1/statuses/user_timeline.json'
with urllib.request.urlopen(url_address) as url:
    data = json.loads(url.read())
print(data)

However, it gives: HTTPError: HTTP Error 400: Bad Request

to me even browser gave {"errors":[{"code":215,"message":"Bad Authentication data."}]} — Learner
– Learner, Commented Dec 1, 2015 at 6:51
The url is of course an example, it should show: {"errors":[{"message":"The Twitter REST API v1 is no longer active. Please migrate to API v1.1. https://dev.twitter.com/docs/api/1.1/overview.","code":64}]} — Joost Döbken
– Joost Döbken, Commented Dec 1, 2015 at 6:53
To read json from url use requests that is best suited also convert the url address to ip address and add arguments to it. For further description read docs at (docs.python-requests.org/en/latest ) — Prashant Shukla
– Prashant Shukla, Commented Dec 1, 2015 at 6:54

irakli khitarishvili · Accepted Answer · 2015-12-01 07:44:07Z

2

twitter is returning 400 code(bad request) so exception is firing. you can use try...except in your code sample and read exception body to get response json {"errors":[{"code":215,"message":"Bad Authentication data."}]}

import json
import urllib.request
from urllib.error import HTTPError


try:

    url_address ='https://api.twitter.com/1.1/statuses/user_timeline.json'
    with urllib.request.urlopen(url_address) as url:
        data = json.loads(url.read())
        print(data)

except HTTPError as ex:
    print(ex.read())

answered Dec 1, 2015 at 7:44

irakli khitarishvili

1092 silver badges5 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Learner · Accepted Answer · 2015-12-01 07:08:00Z

2

To me below code is working- in Python 2.7

import json
from urllib import urlopen
url_address = ['https://api.twitter.com/1.1/statuses/user_timeline.json']
for i in url_address:
    resp = urlopen(i)
    data = json.loads(resp.read())
print(data)

Output-

{u'errors': [{u'message': u'Bad Authentication data.', u'code': 215}]}

If you use requests module-

import requests
url_address = ['https://api.twitter.com/1.1/statuses/user_timeline.json']
for i in url_address:
    resp = requests.get(i).json()
    print resp

Output-

{u'errors': [{u'message': u'Bad Authentication data.', u'code': 215}]}

answered Dec 1, 2015 at 7:08

Learner

5,3001 gold badge29 silver badges39 bronze badges

5 Comments

Joost Döbken Over a year ago

With Python 3 use urllib.request.urlopen. Still this doesnt work for me. Also another example url https://api.github.com/users/mralexgray/repos, I get the error: the JSON object must be str, not 'bytes'

Joost Döbken Over a year ago

data = json.loads(resp.read().decode()) did the job! Can you explain briefly the reason?

Learner Over a year ago

It is just explicitly saying to decode- but it supposed to occur.

Blender Over a year ago

Why are you iterating over a list with one element?

Learner Over a year ago

I just gave a sense in case of multiple element.

Stephen Rauch · Accepted Answer · 2018-03-22 00:14:30Z

0

I prefer aiohttp, it's asynchronous. Last time I checked, urllib was blocking. This means that it will prevent other parts of the script from running while it itself is running. Here's an example for aiohttp:

import aiohttp
async with aiohttp.ClientSession() as session:
async with session.get('https://api.github.com/users/mralexgray/repos') as resp:

    print(await resp.json())

edited Mar 22, 2018 at 0:14

Stephen Rauch♦

50.1k32 gold badges118 silver badges143 bronze badges

answered Mar 21, 2018 at 23:55

user7371019

Collectives™ on Stack Overflow

Python read json from url

3 Answers 3

Comments

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related