AWK doesn't work in Bash Script

Question

I have the following AWK statement in a script:

grep -E $city $DATFILE | awk -F "[\t]+" '($3 >= $minbed) && ($4 >= $minsqft) && ($5 <= $maxprice) && ($6 <= $maxweeks)' $DATFILE | sort -nk5 | less

When I run the script, the output is blank. However, if I run the following:

grep -E Toronto listing.dat | awk -F "[\t]+" '($3 >= 2) && ($4 >= 500) && ($5 <= 900000) && ($6 <= 10)' listing.dat | sort -nk4 | less

It outputs as expected.

I have no idea why this is happening, and I have even replaced the awk statement in the script to echo out the variables to make sure they're passing correctly and they are.

Here is the script thus far:

    #!/bin/bash

    DATFILE=listing.dat

    if [ -f ${DATFILE} ];
    then

            echo -n "Are you looking into anywhere in the GTA, or a specific city?: "
            read uinput

            if [ $uinput == "anywhere" ];
            then
                    echo "You have chosen anywhere"
            elif [ $uinput == "specific" ];
            then
                    echo -n "Which city?: "
                    read city

                    echo -n "Minimum Number of Bedrooms: "
                    read minbed

                    echo -n "Minimum Square Footage (500, 600, etc): "
                    read minsqft

                    echo -n "Maximum Price: "
                    read maxprice

                    echo -n "Maximum Weeks On Market: "
                    read maxweeks

                    echo -n "Sort by (price, sqrft, weeks): "
                    read sortby

                    if [ $sortby == "price" ];
                    then
                            echo -n "Sort by (asc, desc): "
                            read ascdesc
                            if [ $ascdesc == "asc" ];
                            then
                                    grep -E $city $DATFILE | awk -F "[\t]+" '($3 >= $minbed) && ($4 >= $minsqft) && ($5 <= $maxprice) && ($6 <= $maxweeks)' $DATFILE | sort -nk5 | less
                        elif [ $ascdesc == "desc" ];
                        then
                                grep -E $city $DATFILE | awk -F "[\t]+" '($3 >= $minbed) && ($4 >= $minsqft) && ($5 <= $maxprice) && ($6 <= $maxweeks)' $DATFILE | sort -rnk5 | less
                        fi
                fi

        fi

else
        echo "${DATFILE} Not found!"
fi

Can you please help?

Thanks

Your awk script is in single quotes. Shell variables aren't expanded when they are inside signle quotes. So you need to use -v to assign them to an awk variable. — Jerry Jeremiah
– Jerry Jeremiah, Commented Dec 3, 2015 at 23:47
In any case, don't pipe grep results to awk. awk is itself able to filter by regex — hek2mgl
– hek2mgl, Commented Dec 3, 2015 at 23:48
You set the delimiter for awk to "[\t]+" which would match more than one tab. But sort uses any whitespace so you mayneed to set the field separator for sort using -t to the same value you use for awk. — Jerry Jeremiah
– Jerry Jeremiah, Commented Dec 4, 2015 at 1:15

Jonathan Leffler · Accepted Answer · 2015-12-04 00:23:32Z

3

If new to any *nix utility, you can see basic docs using 'man utility_name'; from the man page for 'awk' (man awk):

-v var=val
Assign the value val to the variable var, before execution of the program begins. Such variable values are available to the BEGIN block of an AWK program.

There are other ways to do this, but following the docs your code would be changed to something like:

awk -F "[\t]+" \  
  -v MINBED=$minbed -v MINSQFT=$minsqft -v MAXPRICE=$maxprice -v MAXWEEKS=$maxweeks \
  '($3 >= MINBED) && ($4 >= MINSQFT) && ($5 <= MAXPRICE) && ($6 <= MAXWEEKS)'

edited Dec 4, 2015 at 0:23

Jonathan Leffler

759k145 gold badges961 silver badges1.3k bronze badges

answered Dec 4, 2015 at 0:18

Dale_Reagan

2,0811 gold badge15 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Dave Melia Over a year ago

Okay this is great. It works perfectly, however I cannot get it to filter by city now. Using egrep within the script and piping that through to awk, it shows all cities. If I run the same command out of the script and into the terminal directly, it works.... Any ideas?

Dale_Reagan Over a year ago

remove the '$DATAFILE' from your awk command string; the 'pipe' (match) from grep is sent directly to 'awk' - 'awk' will print everything in the file if you include the file name... [it ignores the data from the 'pipe']...

user.friendly · Accepted Answer · 2015-12-04 07:34:09Z

I'm not able to comment yet due to reputation. Another thing you can do is escape the single quotes around your variable names. If you're still having trouble with all cities showing up, I believe it's because you're supplying awk with the filename. It looks like you need to take that out and let the grep's output be used as input for your awk. I don't have access to my linux box to test it but:

your line (having $DATFILE supplied twice):

grep -E $city $DATFILE | awk -F "[\t]+" '($3 >= $minbed) && ($4 >= $minsqft) && ($5 <= $maxprice) && ($6 <= $maxweeks)' $DATFILE;

option 1: remove the second $DATFILE, escape single quotes

grep -E $city $DATFILE | awk -F "[\t]+" '($3 >= '$minbed') && ($4 >= '$minsqft') && ($5 <= '$maxprice') && ($6 <= '$maxweeks')';

option 2: you may be able to get rid of the grep entirely, with escaped single quotes. awk's IGNORECASE should be a substitute for grep's -E.

awk -F "[\t]+" 'IGNORECASE = 1;/'$city'/&&($3 >= '$minbed')&&($4 >= '$minsqft')&&($5 <= '$maxprice')&&($6 <= '$maxweeks')';

Like I said, can't test it for syntax at the moment; I hope it helps ya.

Happy hacking!

Collectives™ on Stack Overflow

AWK doesn't work in Bash Script

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related