Copy specific rows from csv to csv in Python 2.7

Question

So far I have been trying to copy specific rows including headers from original csv file to a new one. However, once I run my code it was copying a total mess creating a huge document.

This is one of the options I have tried so far, which seems to be the closest to the solution:

import csv
with open('D:/test.csv', 'r') as f,open('D:/out.csv', 'w') as f_out:
     reader = csv.DictReader(f)
     writer = csv.writer(f_out)
     for row in reader:
         if row["ICLEVEL"] == "1":
            writer.writerow(row)

The thing is that I have to copy only those rows where value of "ICLEVEL"(Header name) is equal to "1".

Note: test.csv is very huge file and I cannot hardcode all header names in the writer.

Any demostration of pythonic way of doing this is greatly appreciated. Thanks.

Are you getting any error? What's happening with your solution? — Hussain
– Hussain, Commented Dec 8, 2016 at 6:14
When I run it says "sequence expected" and points at the last row. However, if I go with writer.writerow([row]) it makes a total mess. — Coke
– Coke, Commented Dec 8, 2016 at 6:15

Hussain · Accepted Answer · 2016-12-08 06:31:58Z

4

writer.writerow expects a sequence (a tuple or list). You can use DictWriter which expects a dict.

import csv
with open('D:/test.csv', 'r') as f, open('D:/out.csv', 'w') as f_out:
    reader = csv.DictReader(f)
    writer = csv.DictWriter(f_out, fieldnames=reader.fieldnames)
    writer.writeheader()  # For writing header
    for row in reader:
        if row['ICLEVEL'] == '1':
            writer.writerow(row)

edited Dec 8, 2016 at 6:31

answered Dec 8, 2016 at 6:20

Hussain

5,2176 gold badges48 silver badges72 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Coke Over a year ago

Thanks for an effort, but it gave me a key error ['ICLEVEL'].

Coke Over a year ago

Yes, I just double checked it again :(

Hussain Over a year ago

It should have worked. Could you put few sample rows of your CSV?

Coke Over a year ago

I just replaced my old csv with new one. It worked, however, it copies all the data, not specific one that I need.

Hussain Over a year ago

Passing skipinitialspace=True to the reader might help in that case stackoverflow.com/questions/14885908/…

|

DYZ · Accepted Answer · 2016-12-08 06:18:11Z

1

Your row is a dictionary. CSV writer cannot write dictionaries. Select the values from the dictionary and write just them:

writer.writerow(reader.fieldnames)
for row in reader:
  if row["ICLEVEL"] == "1":
    values = [row[field] for field in reader.fieldnames]
    writer.writerow(values)

answered Dec 8, 2016 at 6:18

DYZ

57.3k10 gold badges73 silver badges101 bronze badges

4 Comments

Coke Over a year ago

You did a great job. Can you suggest how to copy headers as well?

DYZ Over a year ago

The first line of my snippet copies the headers.

DYZ Over a year ago

Works for me in Python 2.7. How does it not work for you?

Coke Over a year ago

Thank you for the solution!

DYZ · Accepted Answer · 2016-12-08 06:40:41Z

0

I would actually use Pandas, not a CSV reader:

import pandas as pd

df=pd.read_csv("D:/test.csv")
newdf = df[df["ICLEVEL"]==1]
newdf.to_csv("D:/out.csv",index=False)

The code is much more compact.

answered Dec 8, 2016 at 6:40

DYZ

57.3k10 gold badges73 silver badges101 bronze badges

3 Comments

Coke Over a year ago

I am not allowed to use Pandas. Your previous answer was correct, I would appreciate if you could restore it. Thanks.

DYZ Over a year ago

Ok, I see. I believe @Hussain's answer is actually better.

Coke Over a year ago

I appreciate his help, however, it didn't filter the data.

Collectives™ on Stack Overflow

Copy specific rows from csv to csv in Python 2.7

3 Answers 3

7 Comments

4 Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

7 Comments

4 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related