Deleting lines in multiple csv files in a folder with Python

Question

I want to delete top and bottom lines in multiple csv files in a folder at once. Then delete the second line after that. My code works well with one file but i have 1000s of files to go through and i cannot specify each file name. How can i modify this to make it work?

#Remove top and bottom lines from all files 
lines = open('My/local/Drive/*.csv').readlines()
open('My/local/Drive/*.csv', 'w').writelines(lines[2:-4]);

#Remove 2nd line from all files
lines = []
with open('My/local/Drive/*.csv', 'r') as f:
    lines = f.readlines()
with open('My/local/Drive/*.csv', 'w') as f:
    f.writelines(lines[:1] + lines[2:]);

could you specify exactly what are the lines you want to remove/keep? — Allan
– Allan, Commented Mar 20, 2019 at 11:30
I am removing the first two lines before the header, the line after the header and the last four lines in the dataset. — enufeffizy
– enufeffizy, Commented Mar 20, 2019 at 11:32
if it were linux I would have solved it in 1 command line, anyway I have answered ;-) — Allan
– Allan, Commented Mar 20, 2019 at 12:12

Allan · Accepted Answer · 2019-03-20 12:10:51Z

1

>>>TAKE A BACKUP OF YOUR FILES BEFORE RUNNING THIS<<<

You can use the following code, where you define path following your needs.

import glob

path='.'
for filename in glob.iglob(path+'/*.csv'):
  with open(filename, 'r') as f:
    lines = f.read().split("\n")
    f.close()
    if len(lines) >= 7:
      lines = lines[2:-5]
      lines = lines[:1] + lines[2:]
      o = open(filename, 'w')
      for line in lines:
        o.write(line+'\n')
      o.close()

in my target folder (2 csv files):

$ ls *.csv
input1.csv  input2.csv

file1:

to delete1
to delete2
ID;TAG;GROUP
todelete;B00;AB0
niub12617500;B01;AB4
niub16371500;B01;AB3
todelete;B00;AB0
todelete;B00;AB0
todelete;B00;AB0
todelete;B00;AB0

file2:

to delete1
to delete2
ID;TAG;GROUP
todelete;B00;AB0
niub12677500;B00;AB2
niub16377500;B01;AB0
todelete;B00;AB0
todelete;B00;AB0
todelete;B00;AB0

After execution:

$ cat input1.csv
ID;TAG;GROUP
niub12617500;B01;AB4
niub16371500;B01;AB3
$ cat input2.csv
ID;TAG;GROUP
niub12677500;B00;AB2
niub16377500;B01;AB0

answered Mar 20, 2019 at 12:10

Allan

12.5k3 gold badges33 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Allan Over a year ago

@enufeffizy: if you are satisfied by my answer, can you vote up and accept? Thank you

M.K · Accepted Answer · 2019-03-20 11:30:53Z

0

Well, you can try and list all files in a directory (having a folder with all your .csv files is a great option here!). This will list you ALL files:

import os
for filename in os.listdir(os.getcwd()):

Or you can just list the .csv files!

import glob
for filename in glob.glob('*.txt'):

Having that, you can do whatever you need to delete those files!

Check this out if you need more insight on listing and operating with that!

Dividing your problem in small problems, everything has an easier solution: how to open every file in a folder

answered Mar 20, 2019 at 11:30

M.K

1,5254 gold badges25 silver badges51 bronze badges

Collectives™ on Stack Overflow

Deleting lines in multiple csv files in a folder with Python

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related