manipulating a csv file and writing its output to a new csv file in python

Question

I have a simple file named saleem.csv which contains the following lines of csv information:

File,Run,Module,Name,,,,,  
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].nic.phy,nbFramesWithInterference,0,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].nic.phy,nbFramesWithoutInterference,0,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].nic.phy,nbFramesWithInterferencePartial,0,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].nic.phy,nbFramesWithoutInterferencePartial,0,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].nic.phy,nbFramesWithInterferenceDropped,0,NaN,NaN,NaN,NaN  
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].nic.phy,nbFramesWithoutInterferenceDropped,0,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].appl,broadcast queued,3,NaN,NaN,NaN,NaN  
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].appl,replies sent,0,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].appl,replies received,0,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].batteryStats,nominal,1.188e+07,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].batteryStats,total,1232.22,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].batteryStats,lifetime,-1,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].batteryStats,Mean power consumption,55.7565,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].batteryStats,num devices,1,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].batteryStats,physical layer,0,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].batteryStats,device total (mWs),1232.22,NaN,NaN,NaN,NaN
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].batteryStats,account,0,1,2,3,4
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].batteryStats,energy (mWs),0,207.519,1024.7,0,0
General-0.sca,General-0-20160706-14:58:51-10463,MyNetwork.node[0].batteryStats,time (s),0,3.83442,18.2656,0,

I want to skip the first line, read this file and only write column[2] and column[4] to a new csv file named out.csv. I have written the following to script to do the job.

import csv

with open('saleem.csv') as f:
    readcsv = csv.reader(f)

    for row in readcsv:
            dele = (row[2], row[4])
            print dele
with open('out.csv', 'w+') as j:
    writecsv = csv.writer(j)
    #for row in dele:
    for row in dele:
            writecsv.writerows(dele)

f.close()
j.close()

This produces the following output:

M,y,N,e,t,w,o,r,k,.,n,o,d,e,[,4,],.,b,a,t,t,e,r,y,S,t,a,t,s
0
M,y,N,e,t,w,o,r,k,.,n,o,d,e,[,4,],.,b,a,t,t,e,r,y,S,t,a,t,s
0

Please help me, Sorry for the mistake previously please, as i mistakenly wrote row.

xgord · Accepted Answer · 2016-07-11 18:49:01Z

2

Edited to reflect revised question

Some problems I can see:

P1: writerows(...)

for row in dele:
            writecsv.writerows(dele)

writerows takes a list of rows to write to the csv file. So it shouldn't be inside a loop where you iterate over all rows and attempt to write them individually.

P2: overwriting

for row in readcsv:
            dele = (row[2], row[4])

You are continuously overwriting dele, so you aren't going to be keeping track of row[2] and row[4] from every row.

What you could do instead:

dele = []
with open('saleem.csv') as f:
    readcsv = csv.reader(f)
    for row in readcsv:
        dele.append([row[2], row[4])
        print([row[2], row[4]])

with open('out.csv', 'w+') as j:
    writecsv.csvwriter(j)
    writecsv.writerows(dele)

This produced output:

MyNetwork.node[0].nic.phy,0
MyNetwork.node[0].nic.phy,0
MyNetwork.node[0].nic.phy,0
MyNetwork.node[0].nic.phy,0
MyNetwork.node[0].nic.phy,0
MyNetwork.node[0].nic.phy,0
MyNetwork.node[0].appl,3
MyNetwork.node[0].appl,0
MyNetwork.node[0].appl,0
MyNetwork.node[0].batteryStats,1.188e+07
MyNetwork.node[0].batteryStats,1232.22
MyNetwork.node[0].batteryStats,-1
MyNetwork.node[0].batteryStats,55.7565
MyNetwork.node[0].batteryStats,1
MyNetwork.node[0].batteryStats,0
MyNetwork.node[0].batteryStats,1232.22
MyNetwork.node[0].batteryStats,0
MyNetwork.node[0].batteryStats,0
MyNetwork.node[0].batteryStats,0

Also, unrelated to your issue at hand, the following code is unnecessary:

f.close()
j.close()

The reason why with open(...): syntax is so widely used, is because it handles gracefully closing the file for you. You don't need to separately close it yourself. As soon as the with block ends, the file will be closed.

edited Jul 11, 2016 at 18:49

answered Jul 11, 2016 at 18:03

xgord

4,8866 gold badges32 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Linux-Fan85 Over a year ago

@xgord Thanks but I have revised the question, I mistakenly wrote the row, but what i actually want is column 2 and column 3 to be written to a new file, please revise your answer, Thanks you once again

xgord Over a year ago

@SaleemRaza edited. In the future, please look over your question before you submit to make sure it's asking what you want it to.

bltpyro · Accepted Answer · 2016-07-11 18:40:27Z

I would suggest using the pandas library.

It makes working with csv files very easy.

import pandas as pd #standard convention for importing pandas

# reads the csv file into a pandas dataframe
dataframe = pd.read_csv('saleem.csv') 

# make a new dataframe with just columns 2 and 4 
print_dataframe = dataframe.iloc[:,[2,4]] 

# output the csv file, but don't include the index numbers or header, just the data
print_dataframe.to_csv('out.csv', index=False, header=False)

If you use Ipython or Jupyter Notebook, you can type

dataframe.head()

to see the first few values of the dataframe. There is a lot more you can do with the library that might be worth learning, but in general it is a great way to read in, filter, and process csv data.

Collectives™ on Stack Overflow

manipulating a csv file and writing its output to a new csv file in python

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related