How to merge more csv files in Python?

Question

I am trying to merge all found csv files in a given directory. The problem is that all csv files have almost the same header, only one column differs. I want to add that column from all csv files to the merged csv file(and also 4 common columns for all csv). So far, I have this:

import pandas as pd
from glob import glob

interesting_files = glob(
    "C:/Users/iulyd/Downloads/*.csv")
df_list = []
for filename in sorted(interesting_files):
    df_list.append(pd.read_csv(filename))
    full_df = pd.concat(df_list, sort=False)
    full_df.to_csv("C:/Users/iulyd/Downloads/merged_pands.csv", index=False)

With this code I managed to merge all csv files, but the problem is that some columns are empty in the first "n" rows, and only after some rows they get their proper values(from the respective csv). How can I make the values begin normally, after the column header?

@GiovaniSalazar here I attached two csv files and their merge result: wetransfer.com/downloads/… — moro_92
– moro_92, Commented Nov 25, 2019 at 10:46

GiovaniSalazar · Accepted Answer · 2019-11-25 21:39:00Z

1

Probably just you need add the name columns :

import pandas as pd
from glob import glob

interesting_files = glob(
    "D:/PYTHON/csv/*.csv")
df_list = []
for filename in sorted(interesting_files):
    print(filename)
    #time,latitude,longitude
    df_list.append(pd.read_csv(filename,usecols=["time", "latitude", "longitude","altitude"]))
    full_df = pd.concat(df_list, sort=False)


print(full_df.head(10))
full_df.to_csv("D:/PYTHON/csv/mege.csv", index=False)

answered Nov 25, 2019 at 21:39

GiovaniSalazar

2,1042 gold badges11 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to merge more csv files in Python?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related