While Loop in Bash Unexpected Character

Question

So I'm writing a script that will take the output of a grep as an array and then iterate a filter over it to output to a file. I'm testing it on my own site and the wget works as expected and generates a list of URLS in the spider.queue. The grep command will also work in terms of filtering by a keyword, but when I add it into a while loop and use an if statement to check if it already exists I'll get the error;

./spider.sh: 19: ./spider.sh: Syntax error: "(" unexpected (expecting "done")

Which would lead me to believe it's a syntax problem with one of the loops.

#!/bin/sh

# Usage - ./spider.sh searchterm www.website.com

## Parameters

search=$1
URL=$2

## Spider WGET

wget -r -e robots=off --header="Accept: text/html" --user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) Gecko/20100101 Firefox/21.0" http://$URL 2>&1 | grep '^--' 2>&1 | awk '{ print $3 }' | grep -v '\.\(css\|js\|png\|gif\|jpg\|JPG\)$' >> spider.queue

## Keyword filter with grep

while true
 do
    PROFILES=($(grep -l -r "$search" $URL))
    for x in ${PROFILES[*]}
    do
            if grep -q $x crawler.queue; then
                    echo "Already Exists"
            else
                    $x >> crawler.queue
            fi
    done
done

You're right I should have that in, missed that thank you. Unfortunately I still get the same error on line 19 — I2obiN
– I2obiN, Commented Feb 22, 2016 at 17:42
Try quoting all your variables, e.g. wget ... "http://$URL" and grep -q "$x" crawler.queue — miken32
– miken32, Commented Feb 22, 2016 at 17:55
Is the code above simplified? Your while true will run for ever. Perhaps there was something wrong with the string that you replaced with true. — Walter A
– Walter A, Commented Feb 22, 2016 at 18:00

chepner · Accepted Answer · 2016-02-22 18:00:11Z

1

/bin/sh does not support arrays, so the syntax error is caused by PROFILES=(....

Switch to #!/bin/bash.

answered Feb 22, 2016 at 18:00

chepner

538k77 gold badges594 silver badges746 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

While Loop in Bash Unexpected Character

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related