Python Api pulling data as dataframe

Question

I'm trying to pull data from an Api which should give me ocean conditions in a certain area. I'm having some trouble to pull the data and separate them into separate variables. Ideally I would like this data to come in as a dataframe but I dont mind it coming another way. I don't have experience doing this so not sure if i'm doing this correctly. My code:

dataLink = 
'http://magicseaweed.com/api/MYApiKEY/forecast/?spot_id=1407&units=eu'
data = urllib.request.urlopen(dataLink)
data = data.readline().decode("utf-8")
data = json.loads(data)
data = pd.DataFrame(data)
swell = data[(data['charts']=='swell')]

Example of URL showing forcast:

[{"timestamp":1502755200,"localTimestamp":1502755200,"issueTimestamp":1502755200,"fadedRating":1,"solidRating":0,"swell":{"absMinBreakingHeight":0.61,"absMaxBreakingHeight":0.95,"unit":"m","minBreakingHeight":0.6,"maxBreakingHeight":0.9,"components":{"combined":{"height":1.2,"period":7,"direction":77.13,"compassDirection":"WSW"},"primary":{"height":1.2,"period":7,"direction":70.75,"compassDirection":"WSW"},"secondary":{"height":0.1,"period":11,"direction":92.74,"compassDirection":"W"}}},"wind":{"speed":18,"direction":90,"compassDirection":"W","chill":13,"gusts":25,"unit":"kph"},"condition":{"pressure":1013,"temperature":15,"weather":12,"unitPressure":"mb","unit":"c"},"charts":{"swell":"https:\/\/hist-1.msw.ms\/wave\/750\/1-1502755200-1.gif","period":"https:\/\/hist-1.msw.ms\/wave\/750\/1-1502755200-2.gif","wind":"https:\/\/hist-1.msw.ms\/gfs\/750\/1-1502755200-4.gif","pressure":"https:\/\/hist-1.msw.ms\/gfs\/750\/1-1502755200-3.gif","sst":"https:\/\/hist-1.msw.ms\/sst\/750\/1-1502755200-10.gif"}},

Your data is not in a format where loading into a data frame would be beneficial to work with. — cs95
– cs95, Commented Aug 16, 2017 at 10:11
Do you have any suggestions on how I should work with the data? I'm not entirely sure what i'm doing in this regard, thanks — Rodrigo Prodohl
– Rodrigo Prodohl, Commented Aug 16, 2017 at 10:18
Hmm.... Does all your data follow this format? Then, it might be worth it. Depends on what you want to do honestly. — cs95
– cs95, Commented Aug 16, 2017 at 10:19
All of the data coming through this website does. I'm trying to implement it into a tkinter GUI so the end user can see the forecast for a given location. There is other data which will also go into the GUI which I'm pulling from incoming log files — Rodrigo Prodohl
– Rodrigo Prodohl, Commented Aug 16, 2017 at 10:22

jezrael · Accepted Answer · 2017-08-16 10:09:58Z

It seems you need json_normalize:

from pandas.io.json import json_normalize

data = json.loads(data)
df = json_normalize(data)
print (df)

                                         charts.wind  condition.pressure  \
0  https:\/\/hist-1.msw.ms\/gfs\/750\/1-150275520...                1013   

   condition.temperature condition.unit condition.unitPressure  \
0                     15              c                     mb   

   condition.weather    ...      swell.maxBreakingHeight  \
0                 12    ...                          0.9   

   swell.minBreakingHeight  swell.unit   timestamp  wind.chill  \
0                      0.6           m  1502755200          13   

   wind.compassDirection wind.direction  wind.gusts  wind.speed  wind.unit  
0                      W             90          25          18        kph  

[1 rows x 38 columns]

Collectives™ on Stack Overflow

Python Api pulling data as dataframe

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related