Python - Upload file to S3 from generated response

Question

The script is fetching pdf from a site (parsing through XML) and I'm trying to auto-upload them to S3 bucket and specific folder. I can get files from URL with specifically generated file-name, but I can't get a line format correctly to upload them to the S3 bucket.

Here is Python Script thus far:



import requests
import xml.etree.ElementTree as ET
import boto3
from botocore.exceptions import NoCredentialsError

url = "https://mylink.com/report"

s3 = boto3.client('s3')


def upload_to_aws(local_file, bucket, s3_file):
    s3 = boto3.client('s3')

    try:
        s3.upload_file(local_file, bucket, s3_file)
        print("Upload Successful")
        return True
    except FileNotFoundError:
        print("The file was not found")
        return False
    except NoCredentialsError:
        print("Credentials not available")
        return False


querystring = {"action":"list"}

headers = {
    'X-Requested-With': "Curl Sample",
    'Authorization': "Basic xxxxxxxxxxxxxxxxxxx",
    'Accept': "*/*",
    'Cache-Control': "no-cache",
    'Host': "api.mylink.com",
    'Accept-Encoding': "gzip, deflate",
    'Connection': "keep-alive",
    'cache-control': "no-cache"
    }

response = requests.request("GET", url, headers=headers, params=querystring)


root = ET.fromstring(response.content)
for child in root.iter(tag='ID'):
 idlist = print(child.text)
 ID=child.text
 querystring_fetch = {"action":"fetch","id":ID}
 response_fetch = requests.request("GET", url, headers=headers, params=querystring_fetch)
 with open(ID + "fetch.pdf","wb") as f:
  f.write(response_fetch.content) # this writes it to local drive 
  upload_to_aws('ID + "fetch.pdf"', 'MY_S3_BUCKET', '%s/%s' %('FOLDER', 'ID + "_fetch.pdf"')) # <--- this line is the key. It's suppose to write to S3, but it does not.

Problem is somewhere with definition of file name in this line:

upload_to_aws('ID + "fetch.pdf"', 'MY_S3_BUCKET', '%s/%s' %('FOLDER', 'ID + "_fetch.pdf"'))

user212514 · Accepted Answer · 2019-12-21 02:49:14Z

1

It looks to me like you have extra single-quotes when constructing the key name. I think the line should be:

key = ID + "fetch.pdf"
upload_to_aws(key, 'MY_S3_BUCKET', '%s/%s' % ('FOLDER', key)) # <--- this line is the key. It's suppose to write to S3, but it does not.

answered Dec 21, 2019 at 2:49

user212514

3,1381 gold badge17 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python - Upload file to S3 from generated response

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related