Writing csv file to Amazon S3 using python

Question

My requirement is to generate csv file and append to a file in Amazon S3. I am able to do it using loop. My code is something like this.

import csv
import random
import simples3
from  simples3 import S3Bucket


quantity=['12','5','13','2','7','6','4','9','15','21','33','62','54','18','26','1']
unitPrice=['1$','2$','3$','4$','5$','6$','7$','8$','9$','10$']



loop= [10]

myfile = open('testcsv.csv', 'ab')
s = S3Bucket('bucket_name',access_key='access_key',secret_key='secret_key')

for a in range(10):

    writing=(random.choice(quantity),random.choice(unitPrice))
    wr =csv.writer(myfile)
    wr.writerow(writing)

    loop.append(writing)    

myfile.close()
print loop
pls = str(loop)
s.put("testconnection.csv",pls,mimetype="text/plain")

Now i want to write to the bucket file within my loop as i did for file in my local file system. I am not getting enough method in S3Bucket to append to a file and put is overwriting the content of file each time the loop executes. Please help me, i am newbie in Python and Amazon S3 world. Thanks in advance.

Roger Jiang · Accepted Answer · 2014-08-21 06:55:57Z

1

It's not possible to append to existing S3 object. You will have to upload the entire content and replace the old object.

answered Aug 21, 2014 at 6:55

Roger Jiang

663 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

piyush pankaj Over a year ago

Thanks Roger for your answer. I will wait for sometime and see if anyone suggest any work around for this. Else i will accept your answer.

piyush pankaj Over a year ago

I have accepted this answer because i myself not sure whether we can do it or not. Anyone please let me know if it is possible. Thankyou.

Collectives™ on Stack Overflow

Writing csv file to Amazon S3 using python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related