How can I add columns from 2 different files to an output in CSV python

Question

my test.csv

1,1,2
2,1,3
3,1,4

my test2.csv

2,3
2,3
2,3

How can i make the output.csv:

1,1,2,2,3
2,1,3,2,3
3,1,4,2,3

so to combine two csv files to one?

Here's my code

import csv, os, sys
with open('test.csv', 'rb') as input, open('output.csv', 'wb') as output, open ('test2.csv', 'rb') as input2:
        reader = csv.reader(input, delimiter = ',')
        reader2 = csv.reader(input2, delimiter = ',')
        writer = csv.writer(output, delimiter = ',')

        all = []                                        
        header = next(reader)
        all.append(header)
        count = 0
        for row,row2 in reader and reader2:
                count += 1
                while count:
                        all.append(row+row2)
                        break
        writer.writerows(all)

Obviously this doesn't work, but does anybody understand what I'm going for?

Ashwini Chaudhary · Accepted Answer · 2014-04-29 11:23:40Z

3

Use zip() to iterate over both readers at once:

reader1 = csv.reader(input, delimiter = ',')
reader2 = csv.reader(input2, delimiter = ',')

for row1, row2 in zip(reader1, reader2):
    writer.writerow(row1 + row2)

Or a shorter version:

writer.writerows(map(list.__add__, row1, row2))

In case the files are huge then using map, zip is not going to be a good idea in Python 2, as they will load all the rows from both files, better go for their iterator versions present in itertools module: itertools.imap and itertools.izip:

for row,row2 in reader and reader2: is equivalent to iterating over just reader2 because and works like this:

>>> 1 and 2 
2
>>> 2 and 3
3
>>> 0 and 2  # returned the first falsy value, but as an iterator is not a falsy value
0            # so it will return `reader2` in your case.

Update:

To update test2.csv in-place you can use the fileinput module, but with this you won't be able to use the csv module.

>>> import fileinput
>>> with open('test.csv') as f:
    for line in fileinput.input('test2.csv', inplace=True):
        print next(f).rstrip() + ',' + line,
...         
>>> !cat test2.csv
1,1,2,2,3
2,1,3,2,3
3,1,4,2,3

Using csv module you'll have to read all the lines from test2.csv in memory first and then write the new data into it.

with open('test.csv') as f1, open('test2.csv', 'r+') as f2:
                                   #open in r+ mode
   reader1 = csv.reader(f1)
   rows_f2 = list(csv.reader(f2)) #read all the rows
   f2.truncate(0)                 #truncate the file
   writer = csv.writer(f2)
   writer.writerows(map(list.__add__, reader1, rows_f2))

edited Apr 29, 2014 at 11:23

answered Apr 29, 2014 at 10:37

Ashwini Chaudhary

252k60 gold badges478 silver badges519 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user3454635 Over a year ago

Thanks, this worked. I was wondering if this would be possible with only input and output file. E.g. 2 columns from input file(test.csv) added(or appended) to output file(test2.csv) without creating a third file.

Kei Minagawa · Accepted Answer · 2014-04-29 11:04:45Z

0

Simply just concatenate line by line with comma...

with open('test.csv', 'rb') as input, open('output.csv', 'wb') as output, open ('test2.csv', 'rb') as input2:
    for row, row2 in zip(input, input2):
        output.write(row.rstrip('\n') + ',' + row2)

answered Apr 29, 2014 at 11:04

Kei Minagawa

4,5515 gold badges31 silver badges43 bronze badges

Collectives™ on Stack Overflow

How can I add columns from 2 different files to an output in CSV python

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related