Shell Script While Loop and Perl Printing Extra Line

Question

I modified a script which a poster gave me from another board to better suit my needs. InputConfig.txt contains directories to find files in, the inbound file age (second column) and the outbound file age (third column). These inbound/outbound numbers for each directory don't have to be the same, I just made them so. Most important is VI and AB directories have specific age to check against, everything else uses the generic 30 minutes.

Perl statement purpose is to capture the timestamp of each file found. The problem is the printf is putting an extra line because the while loop is reading 3 lines but I only need the 2 lines (or however many) to print.

I don't know Perl well enough to fix it - if the problem is with Perl.

Appreciate the help.

InputConfig.txt

/home/MF/NA/CD 30 30
/home/MF/NA/CD/VI 10 10
/home/MF/NA/CD/AB 15 15

Script

#!/bin/ksh

VI=*/CD/VI/*
AB=*/CD/AB/*

cat InputConfig.txt | while read DIR IT OT; do
TS=$(find "${DIR}" -type f -path "${DIR}/*/inbound/*" -mmin "+${IT}" ! -path "${VI}" ! -path "${AB}")
TS=$(find "${DIR}" -type f -path "${DIR}/*/outbound/*.done" -mmin "+${OT}")
TS=$(find "${DIR}" -type f -path "${DIR}/inbound/*" -mmin +"${IT}")
perl -e 'printf("%s,%d\n", $ARGV[0], (stat("$ARGV[0]"))[9]);' "$TS" 
done

Output:

,0
/home/MF/NA/CD/VI/inbound/vis,1492716168
/home/MF/NA/CD/AB/inbound/abc,1492716485

Desired Output

/home/MF/NA/CD/VI/inbound/vis,1492716168
/home/MF/NA/CD/AB/inbound/abc,1492716485

It seems that in the first iteration of your loop $TS is empty? So output from perl -e is ",0"... Also, why do you expect two lines of output when InputConfig.txt has 3 lines — Håkon Hægland
– Håkon Hægland, Commented Apr 21, 2017 at 14:42
You have three lines of inputs, and you ask perl to print a line for each of those... Are you simply missing if length($ARGV[0])? — ikegami
– ikegami, Commented Apr 21, 2017 at 14:50
@HåkonHægland I expect 2 lines of output because those are the files which 'find' found. The first line is not relevant to me. — ocbit
– ocbit, Commented Apr 21, 2017 at 14:59
@ikegami I want Perl to print lines for only files which were found. I understand it will print 3 lines because of the input but is there a way to make it only print for what the find command returned? — ocbit
– ocbit, Commented Apr 21, 2017 at 15:01
Simply don't understands. Here are 3x assignments TS=$(..) in row, so the only last one will be used. What's the point of the first two find? — clt60
– clt60, Commented Apr 21, 2017 at 15:09

clt60 · Accepted Answer · 2017-04-21 15:58:44Z

The script has many problems:

it assigns 3x in row the TS variable, so, only the last one will be used. The first two runs of the find is pointless - so probably you want achieve something else.
youre getting the mtime using perl. It is cool idea if you will read the filenames from the stdin and not starting perl X times. In such case will be faster to use the stat shell command - with other words, you want read the filenames from the stdin.
always use read -r (unless you know why do not want the -r) :)
useless use of cat. Just redirect the whole loop input from a file

So, the script could probably looks like:

#!/bin/ksh

VI=*/CD/VI/*
AB=*/CD/AB/*

while read -r DIR IT OT; do
    find "${DIR}" -type f -path "${DIR}/*/inbound/*" -mmin "+${IT}" ! -path "${VI}" ! -path "${AB}" -print
    find "${DIR}" -type f -path "${DIR}/*/outbound/*.done" -mmin "+${OT}" -print
    find "${DIR}" -type f -path "${DIR}/inbound/*" -mmin +"${IT}" -print
done < InputConfig.txt | perl -lne 'printf "%s,%d\n", $_, (stat($_))[9];'

This is more ksh and/or shell question as perl. :)

BOC · Accepted Answer · 2017-04-21 15:38:32Z

0

If I have well understood, you want to use perl to display the name and the size of the files found by previous find commands. Something like that should work:

#!/bin/ksh

VI=*/CD/VI/*
AB=*/CD/AB/*

cat InputConfig.txt | while read DIR IT OT; do
(find "${DIR}" -type f -path "${DIR}/*/inbound/*" -mmin "+${IT}" ! -path "${VI}" ! -path "${AB}" ;
find "${DIR}" -type f -path "${DIR}/*/outbound/*.done" -mmin "+${OT}" ;
find "${DIR}" -type f -path "${DIR}/inbound/*" -mmin +"${IT}") |
xargs -l perl -e 'printf("%s,%d\n", $ARGV[0], (stat("$ARGV[0]"))[9]);'
done

answered Apr 21, 2017 at 15:38

BOC

1,14910 silver badges20 bronze badges

Comments

ocbit · Accepted Answer · 2017-04-27 14:01:52Z

Thanks everyone for your input but I went back to my original if-else method of script since my TIBCO project from which I am calling the script was not liking the output format.

My script, invoked like ./CDFindFiles /home/NA/CD/:

#!/bin/ksh

FOLDER=$1
VI=*/CD/VI/inbound
AB=*/CD/AB/inbound

find "$FOLDER" -type f -path "${FOLDER}*/inbound/*" -o -path "${FOLDER}*/outbound/*.done" | while read line;
 do
  MODTIME=$(perl -e 'printf "%d\n",(-M shift)*24*60' "$line")
  if [[ "$line" == *"$VI"* && "$MODTIME" -gt 90 || "$line" == *"$AB"* && "$MODTIME" -gt 180 ]]; then
   perl -e 'printf("%s,%d\n", $ARGV[0], (stat("$ARGV[0]"))[9]);' "$line"
  elif [[ "$line" != *"$VI"* && "$line" != *"$AB"* &&  "$MODTIME" -gt 30 ]]; then
    perl -e 'printf("%s,%d\n", $ARGV[0], (stat("$ARGV[0]"))[9]);' "$line"
   fi
 done

Collectives™ on Stack Overflow

Shell Script While Loop and Perl Printing Extra Line

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related