python upload data, not file, to s3 bucket

Question

I know how to upload a file to s3 buckets in Python. I am looking for a way to upload data to a file in s3 bucket directly. In this way, I do not need to save my data to a local file, and then upload the file. Any suggestions? Thanks!

FYI, most issues I see on StackOverflow is when people try to do it this way. It is normally much safer to upload_file() rather than send data to a file. — John Rotenstein
– John Rotenstein, Commented Dec 17, 2018 at 20:25
@John Rotenstein. Do you know why uploading a file is safer? — Quan Zhou
– Quan Zhou, Commented Dec 17, 2018 at 20:55
I think people make more mistakes where code is more complex. It is simpler to write code that creates a file, then uploads a file. When people try to do both in the same step, they run into more problems. It is also easier to test code that does it in separate steps. However, @CharlesLandau's method will probably work (note that it uses the Resource method instead of the Client method). — John Rotenstein
– John Rotenstein, Commented Dec 17, 2018 at 21:05
@JohnRotenstein I am editing to include client method, I thought it would be fine to just point out one — Charles Landau
– Charles Landau, Commented Dec 17, 2018 at 22:21

Charles Landau · Accepted Answer · 2018-12-17 22:22:46Z

1

AFAIK standard Object.put() supports this.

resp = s3.Object('bucket_name', 'key/key.txt').put(Body=b'data')

Edit: it was pointed out that you might want the client method, which is just put_object with the kwargs differently organized

client.put_object(Body=b'data', Bucket='bucket_name', Key='key/key.txt')

edited Dec 17, 2018 at 22:22

answered Dec 17, 2018 at 19:53

Charles Landau

4,2751 gold badge13 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

python upload data, not file, to s3 bucket

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related