User Data script to call aws cli

Question

I am trying to get some files from S3 on startup in an EC2 instance by using a User Data script and the command

/usr/bin/aws s3 cp ...

The log tells me that permission was denied and I believe it is due to aws cli not finding any credentials when executing the user data script.

Running the command with sudo after the instance has started works fine.

I have run aws configure both with sudo and without.

I do not want to use cronjob to run something on startup since I am working with an AMI and often need to change the script, therefore it is more convenient for me to change the user data instead of creating a new AMI everytime the script changes.

If possible, I would also like to avoid writing the credentials into the script.

How can I configure awscli in such a way that the credentials are used when running a user data script?

helloV · Accepted Answer · 2017-12-10 16:06:22Z

3

I suggest you remove the AWS credentials from the instance/AMI. Your userdata script will be supplied with temporary credentials when needed by the AWS metadata server.

See: IAM Roles for Amazon EC2

Clear/delete AWS credentials configurations from your instance and create an AMI
Create a policy that has the minimum privileges to run your script
Create a IAM role and attach the policy you just created
Attach the IAM role when you launch the instance (very important)
Have your userdata script call /usr/bin/aws s3 cp ... without supplying credentials explicitly or using credentials file

answered Dec 10, 2017 at 16:06

helloV

52.8k7 gold badges146 silver badges151 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

AJB · Accepted Answer · 2017-12-11 04:28:42Z

1

You can configure your EC2 instance to receive a pre-defined IAM Role ~~that has its credentials "baked-in" to the instance~~ that it fetches upon instantiation, which it turn will allow it to call aws-cli commands in your User Data script without the need to configure credentials at all.

Here's more info on IAM Roles for EC2: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

It's worth noting here that you'll need to attach the appropriate policies to the IAM Role that you assign to your instance in order for the aws-cli commands to succeed. More information on that can be found here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#working-with-iam-roles

edited Dec 11, 2017 at 4:28

answered Dec 10, 2017 at 12:21

AJB

7,66015 gold badges61 silver badges91 bronze badges

2 Comments

helloV Over a year ago

IAM credentials are never baked-in to the instance. They are fetched on demand from the metadata server.

AJB Over a year ago

@helloV Good point. Edited above. Anyone that wanders in here should see helloV's answer for a more succinct, detailed explanation.

Collectives™ on Stack Overflow

User Data script to call aws cli

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related