Python Pandas - Write New CSV Header Row without Reading/ReWriting Entire File

Question

I have a 27GB CSV file and I want to simply rename the header rows. Can I do this without reading the entire file into a dataframe and then writing the entire file again?

This is essentially what I want to do, but without re-writing the whole 27GB file.

data = pd.read_csv(filename,sep="|",nrows=2)
data.head()

LOC_ID  UPC FW  BOP_U   BOP_$
0   17  438531560821    201712  1   40.0
1   239 438550152328    201719  2   28.8


data.columns = ['WHSE','SKU','PERIOD','QUANTITYONHAND','DOLLARSONHAND']
data.head()


   WHSE           SKU  PERIOD  QUANTITYONHAND  DOLLARSONHAND
0    17  438531560821  201712               1           40.0
1   239  438550152328  201719               2           28.8

So you want to change the header in the file, on the file system? — Chris
– Chris, Commented Feb 17, 2017 at 15:24
There are certainly easier ways to do this than Pandas or even Python. — miradulo
– miradulo, Commented Feb 17, 2017 at 15:25
This is best suited for commandline-like, shell script, instead of using python/pandas just for this. — Zero
– Zero, Commented Feb 17, 2017 at 15:40

miradulo · Accepted Answer · 2017-02-17 15:37:22Z

1

Just specify there being only a single row with nrows.

header_df = pd.read_csv('my_file.csv', index_col=0, nrows=1)

As for re-writing the file, I don't think you'll get around having to process the entire file to re-write.

edited Feb 17, 2017 at 15:37

answered Feb 17, 2017 at 15:19

miradulo

29.8k7 gold badges86 silver badges97 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python Pandas - Write New CSV Header Row without Reading/ReWriting Entire File

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related