how do create a pandas data frame from a string output in python

Question

there are bunch of rest api request, where each request have similiar data. Once I call the url and parse the data that I want, I want to create pandas data frame, create one big data frame after I am done with all of the requests. So data string below should be appended to a data frame:

   #this code is to call each url and get the data 
    hyperId=['hyper1', 'hyper2','hyper3']
    #creating empty data frame with column names
    df1=pd.DataFrame(columns = ['id','servername', 'modelname'

    for id in hyperId1:
        hyperUrl="http://testabc"+id
        resp = requests.get(hyperurl)
        data1=id+","+resp['servername']+","+resp['model']

        #output of each request
        print(data1)
        hyper101,serverabc,proliant

I need to append data1 to data frame called df1 as below

df1 = pd.read_csv(io.StringIO(data1))
df1=df1.append(data)

I am very new to python and pandas. When I run this it says empty data frame with everything appended to the column including column names and actual data. Any help would be appreciated.

Create an empty list, make data1 a list and append it to the empty list. You would then end up with a list of lists which you could use with pd.DataFrame to create a data frame. — norie
– norie, Commented Jul 28, 2021 at 16:05
Just read all the data from the API into a list, then finally create the dataframe out of it using df = pd.DataFrame(dataList, columns= ['id','servername', 'modelname']) — ThePyGuy
– ThePyGuy, Commented Jul 28, 2021 at 16:05

ThePyGuy · Accepted Answer · 2021-07-28 16:15:52Z

1

Don't create any dataframe upfront, and also don't convert and concatenate the string values (since it may have performance impact for a large data). Just create an empty list to hold each rows (as a list), and append each of the list in a loop, then finally create the dataframe, once you have required list of lists.

import  requests
# this code is to call each url and get the data 
hyperId = ['hyper1', 'hyper2', 'hyper3']
# Don't create any Data Frame
# df1 = pd.DataFrame(columns=['id', 'servername', 'modelname'])
dataList = []
for id in hyperId:
    hyperUrl = "http://testabc" + id
    resp = requests.get(hyperUrl)
    # data1 = id + "," + resp['servername'] + "," + resp['model']
    dataList.append([id, resp['servername'], resp['model']])

df = pd.DataFrame(dataList, columns=['id', 'servername', 'modelname'])

answered Jul 28, 2021 at 16:15

ThePyGuy

18.5k5 gold badges24 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

how do create a pandas data frame from a string output in python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related