I thought dsh(Distributed Shell) will be a good option but when I ran my shell script across my nodes, I didn't get my expected output,
dsh -aM -c bash /home/cloudera/bash_script.sh
[email protected]: files in folder
[email protected]: server2
[email protected]: server1
[email protected]: sleep time................
[email protected]: server3
[email protected]: sleep time................
[email protected]: sleep time................
bash_script.sh
#!/bin/bash
while true;
do
shopt -s nullglob
#shopt -s dotglob # To include hidden files
files=(/home/cloudera/MyFolder/*)
echo "files in folder" $files[@]
if [ ${#files[@]} -gt 0 ];
then
for entry in "/home/cloudera/abc"/*
do
cp $entry /home/cloudera/Backup
var=`basename $entry`
var1=`echo ${var//[.csv]/}`
echo $var1
gawk -f abc.awk $entry
rm -r -f $entry
done
fi
echo "server2"
sleep 5s
echo "sleep time................"
sleep 10s
done
My script is working properly if I run it without dsh; why this abnormal behavior? By default, dsh can run up to 64 commands in parallel. Does dsh support all bash commands? What is the best option to go for parrallel processing of shell script accross nodes?
