How to avoid column header repeating in pandas dataframe when forloop placed inside while loop in python

Question

code:

i = 0
while i<3:
    def avoid_repeat_df():
        rows = []
        for i in range(1):
            try:
                rows. append([i, i + 2])
                df = pd. DataFrame(rows, columns=["Aaa", "Bee"])
                df.to_csv('example.csv')#mode='a' write the ouput same as df else only last occurance
                print(df)  
            except:
                pass  
    avoid_repeat_df()        
    i+=1

it prints output as

 Aaa  Bee
0    0    2
   Aaa  Bee
0    0    2
   Aaa  Bee
0    0    2

But in csv only last line writes (in append mode same as df output all the line)

What i expect : avoid repeating column header

 Aaa  Bee
0    0    2
0    0    2  
0    0    2

Hope i explained in details. Thanks for your time and help.

What is your overall goal? Why would you not create your dataframe first and then export it to csv? This seems overly complex. — filiabel
– filiabel, Commented Jul 13, 2021 at 8:30

Anurag Dabas · Accepted Answer · 2021-07-13 08:32:09Z

1

Try defining rows variable(list) outside of the while loop:

i = 0
rows = []
while i<3:
    def avoid_repeat_df():
        for i in range(1):
            try:
                rows. append([i, i + 2])
                df = pd. DataFrame(rows, columns=["Aaa", "Bee"])
                df.to_csv('example.csv')#mode='a' write the ouput same as df else only last occurance
                print(df)  
            except:
                pass  
    avoid_repeat_df()        
    i+=1

Now If you check example.csv:

,Aaa,Bee
0,0,2
1,0,2
2,0,2

answered Jul 13, 2021 at 8:32

Anurag Dabas

24.3k9 gold badges25 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ohm Dios Over a year ago

Thanks, Works fine but when printing df it prints cumulative rows like starts from 1 and keeps on adding lines each iterate. But csv fine. That's solved my issue. Thanks

Rishin Rahim · Accepted Answer · 2021-07-13 08:38:14Z

1

try this:

i = 0
rows = []
while i<3:
    l = [[k, k + 2] for k in range(1)]
    rows.extend(l)
    i+=1
df = pd.DataFrame(rows,columns=["Aaa", "Bee"])
df.to_csv("example.csv")
print(df)

Output

   Aaa  Bee
0    0    2
1    0    2
2    0    2

answered Jul 13, 2021 at 8:38

Rishin Rahim

6554 silver badges14 bronze badges

1 Comment

Ohm Dios Over a year ago

Thanks, I have to use for loop with try and exception. Your answer missed that try and exception loop and df only for try output. Anyhow Thanks for immediate Response.

Sajal · Accepted Answer · 2021-07-13 08:52:18Z

I see 2 ways to handle this, one as mentioned by @Anurag Dabas, to move data frame definition outside the loop. I can help you with the second one where while writing you can simply use header option in to_csv method.

While using mode='a', use header=False, this will skip writing column header each time.

But according to what is presented, the best way would be to use list instead of creating a dataframe, as it is computationally inefficient.

i = 0
rows = list()
while i<3:
    def avoid_repeat_df():
        global rows
        for i in range(1):
            try:
                rows.append([i, i + 2])
            except:
                pass  
    avoid_repeat_df()        
    i+=1
df = pd. DataFrame(rows, columns=["Aaa", "Bee"])
df.to_csv('example.csv')
print(df)

If you still want to create df only

i = 0
while i<3:
    def avoid_repeat_df():
        rows = list()
        for i in range(1):
            try:
                rows.append([i, i + 2])
                df = pd.DataFrame(rows, columns=["Aaa", "Bee"])
                if os.path("example.csv").exists():
                   df.to_csv("example.csv", mode="a", header=False)
                else:
                   df.to_csv("example.csv")
                # This won't change actual df though
                print(df)
            except:
                pass  
    avoid_repeat_df()        
    i+=1

You could also append to df and save when all your iterations have been completed but that would be as good as creating a list and then creating data frame out of it.

Collectives™ on Stack Overflow

How to avoid column header repeating in pandas dataframe when forloop placed inside while loop in python

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related