I have a shell script that pulls parameters from AWS SSM Parameter Store and uses them in some cURL commands to a 3rd party API. If I execute each command manually via the terminal, they all run fine with no issues; however, once I put them in the below script, the cURL command to create the AWS account connector keeps throwing a 500 Internal service error.
Below is a snippet:
#!/bin/bash
# Creating random guid to use for ExternalId
UUID=$(uuidgen)
echo "UUID: $UUID"
# Replacing ExternalId in assumepolicy doc with created guid
$SED_COMMAND -i "s/\"sts:ExternalId\": \".*\"/\"sts:ExternalId\": \"${UUID}\"/g" ./trust-policy.json
# Create new cross-account role with assume role policy
echo "Creating new cross-account role."
aws iam create-role --role-name ${ROLENAME} --assume-role-policy-document file://trust-policy.json
# Attach role policy to newly created role
aws iam put-role-policy --role-name ${ROLENAME} --policy-name ${POLICYNAME} --policy-document file://inline-policy.json
# Grabbing role arn
ARN=$(aws iam get-role --role-name ${ROLENAME} | jq -r '.Role.Arn')
# Logging into SaaS instance; retrieving SID for subsequent API calls
SID=`curl -ks -H "Content-Type: application/json" "Accept: application/json" -X POST "${URL}/rest/authentication/login" -d '{"dsCredentials":{"userName":"'${USER_ID}'","password":"'${PASSWORD}'"}}'`
# Create AWS account connector
echo "Creating AWS account connector."
curl -ks --cookie "sID=${SID}" -H "Content-Type: application/json" "Accept: application/json" -X POST "${URL}/rest/cloudaccounts/aws" -d '{"AddAwsAccountRequest": {"useInstanceRole": false,"crossAccountRole": {"roleArn": "'${ARN}'","externalId": "'${UUID}'"},"workspacesEnabled": false}}'
# Log out of SaaS instance
echo "Logging out."
curl -k -X DELETE ${URL}/rest/authentication/logout?sID=${SID}
The first cURL statement works fine - in the script - which gets a SID for authentication.
SID=`curl -ks -H "Content-Type: application/json" "Accept: application/json" -X POST "${URL}/rest/authentication/login" -d '{"dsCredentials":{"userName":"'${USER_ID}'","password":"'${PASSWORD}'"}}'`
However, the second cURL statement keeps throwing a 500 when run via the shell script: {"error":{"message":"Internal server error"}}
curl -ks --cookie "sID=${SID}" -H "Content-Type: application/json" "Accept: application/json" -X POST "${URL}/rest/cloudaccounts/aws" -d '{"AddAwsAccountRequest": {"useInstanceRole": false,"crossAccountRole": {"roleArn": "'${ARN}'","externalId": "'${UUID}'"},"workspacesEnabled": false}}'
If I take the exact same cURL command and execute manually in a terminal, it works fine. I am also able to make this work in Postman.
In Postman, I find that if a piece of the data in the request is malformed or empty, I get the same "Internal server error" returned, so my hunch is that something is wrong with the request body; however, I can run the exact curl command output via bash -x on a terminal, and it works.
For example, here is output from running the above script with this command: bash -x ./cloudConnectorAutomation.sh
+ echo 'Creating AWS account connector.'
Creating AWS account connector.
+ curl -ks --cookie sID=XXXX -H 'Content-Type: application/json' 'Accept: application/json' -X POST https://XXXX/rest/cloudaccounts/aws -d '{"AddAwsAccountRequest": {"useInstanceRole": false,"crossAccountRole": {"roleArn": "arn:aws:iam::XXXXXXXXXXXX:role/XXXX","externalId": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"},"workspacesEnabled": false}}'
{"error":{"message":"Internal server error"}}
If I copy and paste that exact same curl command straight from the output in the terminal and execute it, it executes perfectly.
I took note of the curl -v output for both the command that ran in the script vs. the terminal, and they were identical except for differing SIDs (due to different authentication sessions). The one that ran in the script shows this as a status code: < HTTP/1.1 500, and the one that I ran manually shows this as the status code: < HTTP/1.1 200.
In a last-ditch effort, I even tested with calling a python script instead of using cURL. Python script below. It is called in the place of the offending cURL request in the initial script at the beginning of this post:
import requests
import sys
import json
SID = sys.argv[1]
URL = sys.argv[2]
ARN = sys.argv[3]
UUID = sys.argv[4]
# Do not proceed if required parameters are empty
if not SID or not URL or not ARN or not UUID:
print ("\nERROR: Required parameters are empty.")
print ("SID: " + SID)
print ("URL: " + URL)
print ("ARN: " + ARN)
print ("UUID: " + UUID)
exit()
endpoint = URL + '/rest/cloudaccounts/aws'
cookies = {
'sID': SID
}
headers = {
'Content-Type': 'application/json',
'Accept': 'application/json'
}
payload = {'AddAwsAccountRequest': {'useInstanceRole': False,'crossAccountRole': {'roleArn': ARN,'externalId': UUID},'workspacesEnabled': False}}
response = requests.post(endpoint, headers=headers, cookies=cookies, data=json.dumps(payload), verify=False)
It returns the same internal server error:
Creating AWS account connector.
+ python3 ./account_connector.py XXXX https://XXXX.com arn:aws:iam::XXXXXXXXX:role/XXXX XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
{"error":{"message":"Internal server error"}}
However, if I execute this line-by-line in ipython, it works perfectly.
I'm at a loss here, and I've spent over a day trying to figure this out.
Again, I truly believe the issue is that my data block is malformed somehow, but I'm not seeing where it is. I'm also perplexed as to why I can take the exact same cURL command and it work in the terminal. I'm researching if there's any nuance or subtle difference in running a cURL command via the terminal vs. a shell script, but I'm not finding that there is.
Any help on this would be greatly appreciated.
