Combine All CSVs in Folder then Delete First 3 Column using Python

Question

I am trying to combine multiple CSVs in a folder then delete the first 3 columns. My code combines the files ok but I can't drop the columns. Can anyone see the issue?

import pandas as pd
import glob

path = r'C:\Users\****' # use your path
all_files = glob.glob(path + "/*.csv")

li = []

for filename in all_files:
    df = pd.read_csv(filename, index_col=None, header=0)
    li.append(df)

frame = pd.concat(li, axis=0, ignore_index=True)

frame.drop(frame.columns[[0, 1, 2]], axis=1)

print(frame)
frame.to_csv('out.csv', index=False)

Can’t you just read_csv without the first 3 columns usecols, with use pd.read_csv('foo.csv', usecols=[3:])? — Prayson W. Daniel
– Prayson W. Daniel, Commented Jun 20, 2021 at 11:55
You need to assign it back, frame = frame.drop(frame.columns[[0, 1, 2]], axis=1) is one way. — Mustafa Aydın
– Mustafa Aydın, Commented Jun 20, 2021 at 12:09

Prayson W. Daniel · Accepted Answer · 2021-06-20 12:22:43Z

2

You can use . iloc[:,3] and select the columns index you desire:


from pathlib import Path
import pandas as pd

path = Path('C:\Users\****') # use your path
all_files = path.glob("*.csv")

frame = pd.concat(
(pd.read_csv(file_, index_col=None, header=0).iloc[:,3:]
for file_ in all_files), axis=0, ignore_index=True)

frame.to_csv('out.csv', index=False)

edited Jun 20, 2021 at 12:22

answered Jun 20, 2021 at 12:06

Prayson W. Daniel

15.8k6 gold badges57 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Prayson W. Daniel Over a year ago

Mmh! In Pandas context, it means all columns minus first 3. Columns are a list of index or actual names

Prayson W. Daniel Over a year ago

I will find a PC. Mustafa, I think you are correct. I read documents and it appears that is is not a list list as order does not matter

Prayson W. Daniel Over a year ago

Okay, change to .iloc[:, 3:] this will read all columns then select all but first three. Everything remains the same

Collectives™ on Stack Overflow

Combine All CSVs in Folder then Delete First 3 Column using Python

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related