S3 upload from python

Question

I want to upload folder in S3, but I have lot of folders which have space between their names. It's an automate process, so could not change the name of the folders manually (from abc d to 'abc d').

folder_name = abc d
import os
cmd = 'aws s3 sync'+''+folder_name+'/'+''+s3path+folder_name+'/'
os.system(cmd)

Are you receiving a specific error? Try print(cmd) to see what it is try to run. You can also use an f-string, such as cmd = f"aws s3 sync '{folder_name}/' '{s3path}{folder_name}/'". — John Rotenstein
– John Rotenstein, Commented Apr 12, 2022 at 7:54
It's working fine if the folder name is not having any space. It will be fine if I could change the folder name. Name=abc d.... To Name='abc d' — Techython
– Techython, Commented Apr 12, 2022 at 8:27
That code can't possibly work. Please post the code you're using when asking questions. As a guess, you need to quote strings in arguments: cmd = 'aws s3 sync "'+folder_name+'/" "'+s3path+folder_name+'/"' — Anon Coward
– Anon Coward, Commented Apr 12, 2022 at 14:49

Lisa Sara Miranda · Accepted Answer · 2022-04-12 06:15:28Z

1

Will be pasting ,my code here. Will be glad if it helps:)

!pip install boto3  ##install boto3
!pip install s3fs  ##required package

import boto3
import pandas as pd

s3 = boto3.client('s3')

s3 = boto3.resource(
service_name='s3',
region_name='us-east-2',
aws_access_key_id='',
aws_secret_access_key='')

# Upload files to S3 bucket
s3.Bucket('bucket_name').upload_file(Filename='foo.csv', Key='foo.csv')
s3.Bucket('bucket_name').upload_file(Filename='bar.csv', Key='bar.csv')

for obj in s3.Bucket('bucket_name').objects.all():
    print(obj)

Note :

the last for loop will print the keys of files that you've uploaded (for confirmation).
aws_access_key_id and aws_secret_access_key - both of these will be available in your aws profile.

answered Apr 12, 2022 at 6:15

Lisa Sara Miranda

111 bronze badge

Sign up to request clarification or add additional context in comments.

1 Comment

Techython Over a year ago

Ohhh great. But suppose I want to change the folder names like if their any space is present then folder name will be inside of single/double quotes

Collectives™ on Stack Overflow

S3 upload from python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related