Build pandas dataframe in for loop

Question

I have a for loop that iterates over a dataframe and calculates two pieces of information:

for id in members['id']
    x = random_number_function()
    y = random_number_function()

I'd like to store id, x and y in a dataframe that is built one row at a time, for each pass through the for loop.

Is there a reason you want to build a dataframe one row at a time, rather than store info in a dict and then turn the dict into a data frame afterward? — iayork
– iayork, Commented Apr 26, 2015 at 22:07
Why can't you just do members['x'] = calculate_x(members['id']) and again for the 'y' column? — EdChum
– EdChum, Commented Apr 26, 2015 at 22:09
You can just modify your calculate_ code to accept a Series so it's vectorised — EdChum
– EdChum, Commented Apr 26, 2015 at 22:10
but you can generate a random array of values using any number of np.random.randint or whatever functions, it's really unnecessary generally to ever loop row-wise — EdChum
– EdChum, Commented Apr 26, 2015 at 22:13

iayork · Accepted Answer · 2015-04-26 22:31:42Z

4

Here's an example of using a dict to build a dataframe:

dict_for_df = {}
for i in ('a','b','c','d'):    # Don't use "id" as a counter; it's a python function
    x = random.random()        # first value
    y = random.random()        # second value
    dict_for_df[i] = [x,y]     # store in a dict
df = pd.DataFrame(dict_for_df) # after the loop convert the dict to a dataframe

answered Apr 26, 2015 at 22:31

iayork

6,7398 gold badges47 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Christopher Jenkins Over a year ago

This works - although I was getting my results transposed the wrong way. Instead of df=pd.DataFrame(dict_for_df) I used, df = DataFrame.from_dict(dict_for_df, orient='index')

iayork Over a year ago

Or just use "pd.DataFrame(dict_for_df).T" -- the "T" transposes the dataframe

Collectives™ on Stack Overflow

Build pandas dataframe in for loop

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related