Upload image to S3 python

Question

I am trying to upload an image to S3 through Python. My code looks like this:

import os
from PIL import Image
import boto
from boto.s3.key import Key

def upload_to_s3(aws_access_key_id, aws_secret_access_key, file, bucket, key, callback=None, md5=None, reduced_redundancy=False, content_type=None):

    conn = boto.connect_s3(aws_access_key_id, aws_secret_access_key)
    bucket = conn.get_bucket(bucket, validate=False)
    k = Key(bucket)
    k.key = key

    k.set_contents_from_file(file)

AWS_ACCESS_KEY = "...."
AWS_ACCESS_SECRET_KEY = "....."

filename = "images/image_0.jpg"
file = Image.open(filename)

key = "image"
bucket = 'images'

upload_to_s3(AWS_ACCESS_KEY, AWS_ACCESS_SECRET_KEY, file, bucket, key)

I am getting this error message:

S3ResponseError: S3ResponseError: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidRequest</Code><Message> The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message> 
<RequestId>90593132BA5E6D6C</RequestId> 
<HostId>...</HostId></Error>

This code is based on the tutorial from this website: http://stackabuse.com/example-upload-a-file-to-aws-s3/

I have tried k.set_contents_from_file as well as k.set_contents_from_filename, but both don't seem to work for me.

The error says something about using AWS4-HMAC-SHA256, but I am not sure how to do that. Is there another way to solve this problem besides using AWS4-HMAC-SHA256? If anyone can help me out, I would really appreciate it.

Thank you!

I think you want to rewrite to use boto3, boto is perhaps no longer supported... — Andy Hayden
– Andy Hayden, Commented May 25, 2018 at 8:31

John Rotenstein · Accepted Answer · 2018-05-26 01:24:25Z

18

Just use:

import boto3

client = boto3.client('s3', region_name='us-west-2')

client.upload_file('images/image_0.jpg', 'mybucket', 'image_0.jpg')

Try to avoid putting your credentials in the code. Instead:

If you are running the code from an Amazon EC2 instance, simply assign an IAM Role to the instance with appropriate permissions. The credentials will automatically be used.
If you are running the code on your own computer, use the AWS Command-Line Interface (CLI) aws configure command to store your credentials in a file, which will be automatically used by your code.

answered May 26, 2018 at 1:24

John Rotenstein

273k28 gold badges456 silver badges541 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

t_sologub Over a year ago

and if you are running the code locally in the Docker container, pass credentials as environment variables.

Raul · Accepted Answer · 2025-04-04 00:04:26Z

In case that you want to upload an image without having to write the image on disk, here is how you can do it with a full working example:

from PIL import Image
import requests
from io import BytesIO
import boto3

# See boto3 for Configuring Credentials
# If you want to test this script you can define your credentials below
# (A best practice is to let boto3 handles the auth)
aws_access_key_id = None 
aws_secret_access_key = None
aws_session_token = None


def image_upload(image, key, bucket_name):
    """Upload a PIL image to s3.

    Parameters
    ----------
    image : PIL.PngImagePlugin.PngImageFile
        The PIL image.
    key : str
        The file key.
    bucket_name : str
        The bucket name.
    """

    client = boto3.client(
        's3',
        aws_access_key_id=aws_access_key_id,
        aws_secret_access_key=aws_secret_access_key,
        aws_session_token=aws_session_token)

    file = BytesIO()
    image.save(file, format=image.format)
    file.seek(0)

    # Upload image to s3
    client.upload_fileobj(
        Fileobj=file,
        Bucket=bucket_name,
        Key=key)


def image_download(key, bucket_name):
    """Download a PIL image stored in s3.

    Parameters
    ----------
    key : str
        The file key.
    bucket_name : str
        The bucket name.

    Return
    -------
    image : PIL.PngImagePlugin.PngImageFile
        The PIL image.
    """

    s3 = boto3.resource(
        's3',
        aws_access_key_id=aws_access_key_id,
        aws_secret_access_key=aws_secret_access_key,
        aws_session_token=aws_session_token)

    bucket = s3.Bucket(bucket_name)
    image = Image.open(
        bucket.Object(key).get()['Body']
    )

    return image

# Rabbit image
img_url = (
    "https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/"
    "Oryctolagus_cuniculus_Tasmania_2_%28cropped%29.jpg/"
    "470px-Oryctolagus_cuniculus_Tasmania_2_%28cropped%29.jpg")

# Get the image as a PIL object
response = requests.get(img_url)
image = Image.open(BytesIO(response.content))

bucket_name = 'my-bucket-name'
key = 'my_image.png'

# Upload to s3
image_upload(
    image=image, 
    key=key, 
    bucket_name=bucket_name)

# Get from s3
image2 = image_download(key, bucket_name)

Collectives™ on Stack Overflow

Upload image to S3 python

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related