Adding Counter in shell script

Try this:

counter=0
while true; do
  if /home/hadoop/latest/bin/hadoop fs -ls /apps/hdtech/bds/quality-rt/dt=$DATE_YEST_FORMAT2 then
       echo "Files Present" | mailx -s "File Present"  -r [email protected] [email protected]
       break
  elif [[ "$counter" -gt 20 ]]; then
       echo "Counter limit reached, exit script."
       exit 1
  else
       let counter++
       echo "Sleeping for another half an hour" | mailx -s "Time to Sleep Now"  -r [email protected] [email protected]
       sleep 1800
  fi
done

Explanation

• break - if files are present, it will break and allow the script to process the files.
• [[ "$counter" -gt 20 ]] - if the counter variable is greater than 20, the script will exit.
• let counter++ - increments the counter by 1 at each pass.

#!/usr/bin/env bash
counter=0
while [[ "$counter" -lt 20 ]] 
do
  if [[ $counter -gt 0 ]] ; then
    echo "Counter: $counter time(s); Sleeping for another half an hour" | mailx -s "Time to Sleep Now"  -r [email protected] [email protected]
    sleep 1800  
  fi
  if /home/hadoop/latest/bin/hadoop fs -ls /apps/hdtech/bds/quality-rt/dt=$DATE_YEST_FORMAT2 ; then
       echo "Files Present" | mailx -s "File Present"  -r [email protected] [email protected]
       exit 0
  fi
  : $((counter++))
done

echo "Counter: $counter times reached; Exiting loop!"
exit 1

Here's another take on this. I've added this bit of code to illustrate these points:

1. The while can take your counter as the condition - this removes the need to check the variable value in the loop.
2. Since the if branches cause the loop to end anyway ( via break or exit ) there is no need for the final else.
3. I've provided a different syntax for increasing the counter. To be honest, I've never seen koola's version (let counter++) but I like it a lot (it's simpler to me).
4. I'm not 100% sure that some of the while loops provide as answers actually loop 20 times. Rather, I think some of them loop 21 times.
5. The solutions presented actually email the admin an extra time - there is no need to tell the admin that you are sleeping for a half an hour if you are going to be exiting anyway. I've added an additional if at the beginning of the while loop to avoid the extra email (and sleep).

Here's the output for a run that fails to find files 20 times:

Counter: 1 time(s); Sleeping for another half an hour
Counter: 2 time(s); Sleeping for another half an hour
...
Counter: 19 time(s); Sleeping for another half an hour
Counter: 20 times reached; Exiting loop!

The short of if is that we check for files 20 times, but only sleep 19 times.

You may do this with a for loop instead of a while:

max_loop=20
for ((count = 0; count < max_loop; count++)); do
  if /home/hadoop/latest/bin/hadoop fs -ls /apps/hdtech/bds/quality-rt/dt=$DATE_YEST_FORMAT2 then
       echo "Files Present" | mailx -s "File Present"  -r [email protected] [email protected]
       break
  else
       echo "Sleeping for half an hour" | mailx -s "Time to Sleep Now"  -r [email protected] [email protected]
       sleep 1800
  fi
done

if [ "$count" -eq "$max_loop" ]; then
  echo "Maximum number of trials reached" >&2
  exit 1
fi

There is a shell tool that produces a list of numbers, but I couldn't remember what it was called. I found question above and its answers when I was searching for that shell tool. Later, I remembered that the shell tool I was looking for is seq. (It may not be available on every system, but mine, with macOS 13.6, has it.)

Here's an implementation of the script using seq…

maxAttempts=20
for i in `seq ${maxAttempts}`; do
  if /home/hadoop/latest/bin/hadoop fs -ls /apps/hdtech/bds/quality-rt/dt=$DATE_YEST_FORMAT2; then
       echo "Files Present" | mailx -s "File Present"  -r [email protected] [email protected]
       exit 0
  else
       echo "Sleeping for half an hour" | mailx -s "Time to Sleep Now"  -r [email protected] [email protected]
       sleep 1800
  fi
done

echo "Failed after ${maxAttempts} attempts."
exit 1