2

i have an empty dataframe[] and want to append additional dataframes using for loop without overwriting existing dataframes, regular append method is overwriting the existing dataframe and showing only the last appended dataframe in output.

3
  • 2
    append won't overwrite the existing dataframe, unless you want it. df.append(df1) won't change df. But df = df.append(df1) will do. Commented Sep 20, 2018 at 23:26
  • i tried 1st option i.e. df.append(df1) within for loop but the output is empty data frame. Commented Sep 21, 2018 at 7:21
  • I try to make an answer. Hope you get the point. Commented Sep 21, 2018 at 8:58

5 Answers 5

1

use concat() from the pandas module.

import pandas as pd
df_new = pd.concat([df_empty, df_additional])

read more about it in the pandas Docs.

regarding the question in the comment...

df = pd.DataFrame(insert columns which your to-be-appended-df has too)

for i in range(10):
    function_to_get_df_new()
    df = pd.concat([df, df_new])
Sign up to request clarification or add additional context in comments.

2 Comments

i want to use append through for loop
this option "df = pd.concat([df, df_new])" is overwriting the existing frames and giving only last df in output
1

Let you have list of dataframes list_of_df = [df1, df2, df3].

You have empty dataframe df = pd.Dataframe()

If you want to append all dataframes in list into that empty dataframe df:

for i in list_of_df:
    df = df.append(i)

Above loop will not change df1, df2, df3. But df will change. Note that doing df.append(df1) will not change df, unless you assign it back to df so that df = df.append(df1)

Comments

1

You can't also use set:

df_new = pd.concat({df_empty, df_additional})

Because pandas.DataFrame objects can't be hashed, set needs hashed so that's why

Or tuple:

df_new = pd.concat((df_empty, df_additional))

They are little quicker...

Update for for loop:

df = pd.DataFrame(data)

for i in range(your number):
    df_new=function_to_get_df_new()
    df = pd.concat({df, df_new}) # or tuple: df = pd.concat((df, df_new))

4 Comments

i want to add additional dfs through Loop
@user10234885 Done!
while doing with 1st (using for loop\ df=pd.concat({df,df_new})) option it is giving me following error: TypeError: 'DataFrame' objects are mutable, thus they cannot be hashed
@user10234885don't use tuple or list for this set is not right sorry fr this
1

The question is already well answered, my 5cts are the suggestion to use ignore_index=True option to get a continuous new index, not duplicate the older ones.

import pandas as pd

df_to_append = pd.DataFrame([[1, 2], [3, 4]], columns=list('AB')) # sample
df = pd.DataFrame() # this is a placeholder for the destination

for i in range(3):
    df = df.append(df_to_append, ignore_index=True)

Comments

0

I don't think you need to use for loop here, try concat()

import pandas
result = pandas.concat([emptydf,additionaldf])

pandas.concat documentation

1 Comment

In for loop i am calling a function which is generating new dfs , those dfs i want to append in existing empty df

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.