User data bash script to start services in EC2 instance not executing

Question

I'm having difficulty getting certain services running within my EC2 instance. I'm using an Amazon Linux AMI, and would like to start neo4j, the graphical database, as a service when my instance first launches.

However, I've been having trouble getting anything start, let alone neo4j. I can't even get the basic tutorial user data script provided by Amazon working. I copied and pasted the below code:

#!/bin/bash
yum update -y
yum install -y httpd24 php56 mysql55-server php56-mysqlnd
service httpd start
chkconfig httpd on

into the User Data section of my EC2 instance, and then started up my EC2 instance. Theoretically, since httpd is started, I should be able to navigate to the IP address of my instance in a browser and see the default Apache web page (I have allowed inbound HTTP at port 80 for security group settings). However, I get a Connection refused error.

This is confirmed when I issue the following commands after SSH-ing into my instance:

sudo su service httpd status

The response is httpd is stopped, when clearly the user data script is supposed to start it. Amazon recommends to debug that I check /var/log/cloud-init-output.log for error messages, but I don't see any when I inspect the log:

Cloud-init v. 0.7.6 running 'init-local' at Sun, 12 Nov 2017 16:41:03 +0000. Up 5.73 seconds.
Cloud-init v. 0.7.6 running 'init' at Sun, 12 Nov 2017 16:41:03 +0000. Up 5.91 seconds.
ci-info: +++++++++++++++++++++++Net device info+++++++++++++++++++++++
ci-info:  Device   Up     Address         Mask          Hw-Address    
ci-info:    lo    True   127.0.0.1     255.0.0.0            .         
ci-info:   eth0   True  172.31.7.27  255.255.240.0  0a:8c:a2:67:1f:46 
ci-info: ++++++++++++++++++++++++++++++Route info+++++++++++++++++++++++++++++++
ci-info:  Route    Destination     Gateway        Genmask      Interface  Flags 
ci-info:    0        0.0.0.0      172.31.0.1      0.0.0.0         eth0      UG  
ci-info:    1    169.254.169.254   0.0.0.0    255.255.255.255     eth0      UH  
ci-info:    2       172.31.0.0     0.0.0.0     255.255.240.0      eth0      U   
Generating public/private rsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_rsa_key.
Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub.
The key fingerprint is:
...
The key's randomart image is:
+---[RSA 2048]----+
[RANDOM ART]
+----[SHA256]-----+
Generating public/private dsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_dsa_key.
Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub.
The key fingerprint is:
...
The key's randomart image is:
+---[DSA 1024]----+
+----[SHA256]-----+
Generating public/private ecdsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_ecdsa_key.
Your public key has been saved in /etc/ssh/ssh_host_ecdsa_key.pub.
The key fingerprint is:
...
The key's randomart image is:
... [RANDOMART IMAGE]
Cloud-init v. 0.7.6 running 'modules:config' at Sun, 12 Nov 2017 16:41:04 +0000. Up 6.70 seconds.
Loaded plugins: priorities, update-motd, upgrade-helper
No packages needed for security; 0 packages available
No packages marked for update
Cloud-init v. 0.7.6 running 'modules:final' at Sun, 12 Nov 2017 16:41:08 +0000. Up 10.93 seconds.
Cloud-init v. 0.7.6 finished at Sun, 12 Nov 2017 16:41:08 +0000. Datasource DataSourceEc2.  Up 11.04 seconds
Cloud-init v. 0.7.6 running 'init-local' at Sun, 12 Nov 2017 17:10:45 +0000. Up 5.85 seconds.
Cloud-init v. 0.7.6 running 'init' at Sun, 12 Nov 2017 17:10:46 +0000. Up 6.04 seconds.
ci-info: +++++++++++++++++++++++Net device info+++++++++++++++++++++++
ci-info:  Device   Up     Address         Mask          Hw-Address    
ci-info:    lo    True   127.0.0.1     255.0.0.0            .         
ci-info:   eth0   True  172.31.7.27  255.255.240.0  0a:8c:a2:67:1f:46 
ci-info: ++++++++++++++++++++++++++++++Route info+++++++++++++++++++++++++++++++
ci-info:  Route    Destination     Gateway        Genmask      Interface  Flags 
ci-info:    0        0.0.0.0      172.31.0.1      0.0.0.0         eth0      UG  
ci-info:    1    169.254.169.254   0.0.0.0    255.255.255.255     eth0      UH  
ci-info:    2       172.31.0.0     0.0.0.0     255.255.240.0      eth0      U   
Cloud-init v. 0.7.6 running 'modules:config' at Sun, 12 Nov 2017 17:10:46 +0000. Up 6.61 seconds.
Cloud-init v. 0.7.6 running 'modules:final' at Sun, 12 Nov 2017 17:10:47 +0000. Up 7.03 seconds.
Cloud-init v. 0.7.6 finished at Sun, 12 Nov 2017 17:10:47 +0000. Datasource DataSourceEc2.  Up 7.11 seconds
Cloud-init v. 0.7.6 running 'init-local' at Sun, 12 Nov 2017 17:18:25 +0000. Up 5.60 seconds.
Cloud-init v. 0.7.6 running 'init' at Sun, 12 Nov 2017 17:18:25 +0000. Up 5.79 seconds.
ci-info: +++++++++++++++++++++++Net device info+++++++++++++++++++++++
ci-info:  Device   Up     Address         Mask          Hw-Address    
ci-info:    lo    True   127.0.0.1     255.0.0.0            .         
ci-info:   eth0   True  172.31.7.27  255.255.240.0  0a:8c:a2:67:1f:46 
ci-info: ++++++++++++++++++++++++++++++Route info+++++++++++++++++++++++++++++++
ci-info:  Route    Destination     Gateway        Genmask      Interface  Flags 
ci-info:    0        0.0.0.0      172.31.0.1      0.0.0.0         eth0      UG  
ci-info:    1    169.254.169.254   0.0.0.0    255.255.255.255     eth0      UH  
ci-info:    2       172.31.0.0     0.0.0.0     255.255.240.0      eth0      U   
Cloud-init v. 0.7.6 running 'modules:config' at Sun, 12 Nov 2017 17:18:26 +0000. Up 6.41 seconds.
Cloud-init v. 0.7.6 running 'modules:final' at Sun, 12 Nov 2017 17:18:26 +0000. Up 6.84 seconds.
Cloud-init v. 0.7.6 finished at Sun, 12 Nov 2017 17:18:26 +0000. Datasource DataSourceEc2.  Up 6.93 seconds

I looked on StackOverflow for similar posts, and found this: user data script doesn't launch with ec2 instance. The accepted answer says that user data scripts only execute on the first boot cycle- does that mean I need to terminate my instance and launch a completely fresh instance in order to execute my script? In which case, I feel like that defeats the whole purpose of a startup script to begin with!

EDIT: I added in a few echo test statements to see if they were being printed out in the logs- nope!

However, I did try curl http://169.254.169.254/latest/meta-data from my EC2 instance, and got back this output:

#!/bin/bash
echo "Testing 1"
yum update -y
echo "Testing 2"
yum install -y httpd24 php56 mysql55-server php56-mysqlnd
echo "Testing 3"
service httpd start
chkconfig httpd on

So it's definitely loading my user data... just not executing it.

also have you enabled ports for yum install ? you can add echo "Test" in your user data section to see if user data section is being invoked or not. #!/bin/bash yum update -y echo "test" yum install -y httpd24 php56 mysql55-server php56-mysqlnd echo "starting httpd service" service httpd start chkconfig httpd on — Usman Azhar
– Usman Azhar, Commented Nov 13, 2017 at 3:23
If there is no echo output in your log, that means your script didnt run during your instance startup. Keep in mind, user-data script just start only one time at the first bootup. You can run other instance with smaller instance type to test your script. — Bui Anh Tuan
– Bui Anh Tuan, Commented Nov 13, 2017 at 4:04
no! Fisrt time when you launch new instance. The next time you stop then restart instance, it will not run user-data again. Thats why you must launch new instance to test user-data script — Bui Anh Tuan
– Bui Anh Tuan, Commented Nov 13, 2017 at 4:09
@Yu Chen - correct. Would you want the installation process to run each time on reboot? If you want to start a process on each reboot, then look into /etc/rc.local. Userdata only runs on the instance launch. Its purpose is to run once, to allow for one-time server configuration before the instance is put into service. — Rodrigo Murillo
– Rodrigo Murillo, Commented Nov 13, 2017 at 4:11

Rodrigo Murillo · Accepted Answer · 2017-11-13 15:49:09Z

From Configuring Instances with User Data

When you specify user data, note the following:

User data is treated as opaque data: what you give is what you get back. It is up to the instance to be able to interpret it.

User data is limited to 16 KB. This limit applies to the data in raw form, not base64-encoded form.

User data must be base64-encoded. The Amazon EC2 console can perform the base64 encoding for you or accept base64-encoded input.

User data must be decoded when you retrieve it. The data is decoded when you retrieve it using instance metadata and the console.

User data is executed only at launch. If you stop an instance, modify the user data, and start the instance, the new user data is not executed automatically

Userdata only runs on the instance launch. Its purpose is to run once, to allow for one-time server configuration tasks before the instance is put into service.

Do you want the software installation process to run each time on reboot? If you want to start a process on each reboot, then look into /etc/rc.local.

user10748367 · Accepted Answer · 2022-01-14 08:33:45Z

I've just encountered a similar problem and the solution that worked for me is described in this article: How can I utilize user data to automatically run a script with every restart of my Amazon EC2 Linux instance?

So, just add your own script after the #!/bin/bash line and put the whole block to the Edit user data of your EC2 instance. Basically, it will look something like this:

Content-Type: multipart/mixed; boundary="//"
MIME-Version: 1.0

--//
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config.txt"

#cloud-config
cloud_final_modules:
- [scripts-user, always]

--//
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="userdata.txt"

#!/bin/bash

# This script is run after the system reboots!

# 1. Go to the project directory.
cd /home/ubuntu/my-webapp

# 2. Run docker compose to launch the app.
docker-compose -f docker-compose.prod.yml up -d
--//--

Here's an example of what that page looks like, and notice the unchecked box for "Input is already base64-encoded":

Collectives™ on Stack Overflow

User data bash script to start services in EC2 instance not executing

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related