How do I save data in python(pandas)

Question

When I save the data to csv file only records with id 103 are saved and records having ids 101 and 102 are not available.

import pandas as pd
import requests as rq

for vehicleList in range(101, 103):
    vehicleList = vehicleList + 1
    x = str(vehicleList)

    r = rq.get('https://api.tfl.gov.uk/vehicle/' + x + '/arrivals')
    r = r.text
    df = pd.read_json(r)

    df.to_csv('filename.csv')

can you add the url complet? I mean in which case x ....101...102...? — GiovaniSalazar
– GiovaniSalazar, Commented Dec 11, 2019 at 22:32
First of all, fix your indentation. Second, it looks like you override your file after each loop. — rafaelc
– rafaelc, Commented Dec 11, 2019 at 22:32
Variable and function names should follow the lower_case_with_underscores style. — AMC
– AMC, Commented Dec 12, 2019 at 1:29

Denver · Accepted Answer · 2019-12-11 22:34:48Z

3

You need to concatenate the data before saving. For example:

import pandas as pd
import requests as rq

temp = pd.DataFrame()
df = pd.DataFrame()

for vehicleList in range(101, 103):
    vehicleList = vehicleList + 1
    x = str(vehicleList)
    r = rq.get('https://api.tfl.gov.uk/vehicle/' + x + '/arrivals')
    r = r.text
    temp = pd.read_json(r)

    # some identifier here  
    temp['Type'] = x

    # concat
    df = pd.concat([df, temp]).reset_index(drop=True)

df.to_csv('filename.csv')

You assign to a temporary DataFrame each loop temp and then add a unique id to denote the group temp['Type'] and then concatenate to df. Once the loop is complete you save df with all iterations included.

answered Dec 11, 2019 at 22:34

Denver

6394 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

istanbul34 Over a year ago

thank you vey much ,I'm new to python so I'm missing so much.

Collectives™ on Stack Overflow

How do I save data in python(pandas)

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related