Running local shell script on multiple remote server in parallel and get the status

I like this form when doing this:

$ cat hosts.txt | xargs -n1 -P8 sh -c 'ssh -T "$1" bash -s < ./hello.bash' sh

Or with {} if you need them:

$ cat hosts.txt | xargs -n1 -P8 -I{} sh -c 'ssh -T "$1" bash -s < ./hello.bash' sh {}

Example

$ cat hello.bash
#!/bin/bash

echo "hi from server: $(hostname)"

$ cat hosts.txt | xargs -n1 -P8 sh -c 'ssh -T "$1" bash -s < ./hello.bash' sh
hi from server: mulder.mydom.com
hi from server: skinner.mydom.com
hi from server: manny.mydom.com

Details:

• -n1 -P8 - tells xargs to take 1 argument as input and run 8 instances of ssh
• sh -c 'ssh -T "$1"
  • this invokes a shell and then runs the command after -c "..."
  • notice we're passing $1 in here, this is to avoid injection attacks.
  • -T disables pseudo terminal. $1 is the contents of the file being cat'ed in.
• bash -s < ./hello.bash' - the commands that will get passed to ssh
• sh - the trailing sh is what gets passed to xargs as the shell to invoke, argument #0 ($0).

References

• rasschaert/xargs-ssh
• Running programs in parallel using xargs
• Is it possible to use `find -exec sh -c` safely?
• Passing multiple parameters via xargs

hello.bash:

doit() {
  echo Define
  echo what you want done in a
  echo function
}
export -f doit

Then do:

. hello.bash
parallel --env doit --slf hosts.txt --tag --nonall doit

Yes, you can do this with AWS Systems manager. AWS Systems Manager Run Command allows you to remotely and securely run set of commands on EC2 as well on-premise server. Below are high-level steps to achieve this.

Attach Instance IAM role: The ec2 instance must have IAM role with policy AmazonSSMFullAccess. This role enables the instance to communicate with the Systems Manager API.

Install SSM Agent: The EC2 instance must have SSM agent installed on it. The SSM Agent process the run command requests & configure the instance as per command.

Execute command : Example usage via AWS CLI: In --instance-ids give the list of ec2 instance ids. Execute the following command to retrieve the services running on the instance. Replace Instance-ID with ec2 instance id.

aws ssm send-command --document-name "AWS-RunShellScript" --comment "listing services" --instance-ids "Instance-ID" --parameters commands="service --status-all" --region us-west-2 --output text

More information here