Python: Looping through multiple csv files and making multiple new csv files

Question

I am starting out in Python, and I am looking at csv files.

Basically my situation is this:

I have coordinates X, Y, Z in a csv.

and I want to go through and add a user defined offset value to all Z values and make a new file with the edited z-values.

here is my code so far which I think is right:

# list of lists we store all data in
allCoords = []
# get offset from user
offset = int(input("Enter an offset value: "))
# read all values into memory
with open('in.csv', 'r') as inFile: # input csv file
    reader = csv.reader(inFile, delimiter=',') 
    for row in reader:
        # do not add the first row to the list
        if row[0] != "X": 
            # create a new coord list
            coord = []
            # get a row and put it into new list
            coord.append(int(row[0]))
            coord.append(int(row[1]))
            coord.append(int(row[2]) + offset)
            # add list to list of lists
            allCoords.append(coord)

# write all values into new csv file
with open(in".out.csv", "w", newline="") as f:
    writer = csv.writer(f)
    firstRow = ['X', 'Y', 'Z']
    allCoords.insert(0, firstRow)
    writer.writerows(allCoords)

But now come's the hard part. How would I go about going through a bunch of csv files (in the same location), and producing a new file for each of the csv's.

I am hoping to have something like: "filename.csv" turns into "filename_offset.csv" using the original file name as a starter for the new filename, appending ".offset" to the end.

I think I need to use "os." functions, but I am not sure how to, so any explanation would be much appreciated along with the code! :)

Sorry if I didn't make much sense, let me know if I need to explain more clearly. :)

Thanks a bunch! :)

module glob would help in this situation

The6thSense
– The6thSense

2015-10-05 08:47:09 +00:00
Commented Oct 5, 2015 at 8:47 — The6thSense
– The6thSense, Commented Oct 5, 2015 at 8:47

LetzerWille · Accepted Answer · 2015-10-05 10:40:22Z

2

shutil.copy2(src, dst)¶
Similar to shutil.copy(), but metadata is copied as well

shutil

The glob module finds all the pathnames matching a specified pattern
according to the rules used by the Unix shell. No tilde expansion is
done, but *, ?, and character ranges expressed with [] will be correctly matched

glob

import glob
from shutil import copy2
import shutil
files = glob.glob('cvs_DIR/*csv')

for file in files:

    try:
        # need to have full path of cvs_DIR
        oldName = os.path.join(cvs_DIR, file)
        newName = os.path.join(cvs_DIR, file[:4] + '_offset.csv')
        copy2(oldName,newName)

    except shutil.Error as e:
        print('Error: {}'.format(e))

edited Oct 5, 2015 at 10:40

answered Oct 5, 2015 at 9:10

LetzerWille

5,6965 gold badges26 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

FactorV Over a year ago

Awesome, thanks Letzer. Quick question so I understand. In the newName variable, what is the "[:4]" for?

LetzerWille Over a year ago

@FactorV file[:4] it removes last 4 chars from the string. it will remove .csv, since we need to append _offset.csv

FactorV Over a year ago

great, thanks for that. its a handy little thing to know. :)

Community · Accepted Answer · 2017-05-23 11:54:34Z

1

BTW, you can write ...

for row in reader:
    if row[0] == "X":
        break
for row in reader:
    coord = []
    ...

... instead of ...

for row in reader:
    if row[0] != "X": 
        coord = []
        ...

This stops checking for 'X'es after the first line. It works because you dont work with a real list here but with a self consuming iterator, which you can stop and restart.

See also: Detecting if an iterator will be consumed.

edited May 23, 2017 at 11:54

CommunityBot

11 silver badge

answered Oct 5, 2015 at 9:45

user1658543

1 Comment

FactorV Over a year ago

awesome, thanks Nils, that makes sense and reads a little better to :)

Collectives™ on Stack Overflow

Python: Looping through multiple csv files and making multiple new csv files

2 Answers 2

3 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related