Bash script with regex not behaving on Ubuntu

Question

I have a Bash script that is working on my OpenSuSE box, but when copied across to my Ubuntu box, is not working. The script reads in from a file. The file has fields separated by white space (tabs and spaces).

#!/bin/bash
function test1()
{
    while read LINE
    do
        if [[ $LINE =~ "^$" || $LINE =~ "^#.*" ]] ; then
            continue;
        fi
        set -- $LINE
        local field1=$1
        local field2=$2
    done < test.file
}

test1

with test.file containing:

# Field1Header    Field2Header
abcdef            A-2
ghijkl            B-3

There seem to be two problems:

(1) $field2, the one with the hyphen, is blank

(2) The regex to strip out the blank lines and lines that start with # is not working

Anyone know what's wrong? As I said, it works fine on OpenSuSE.

Thanks, Paul

Are you actually running it in bash? On Ubuntu, /bin/sh is not bash. — William Pursell
– William Pursell, Commented Jul 3, 2012 at 15:01
Hi. At the top of the file I have "#!/bin/bash". I'll edit my post to that effect. — user265330
– user265330, Commented Jul 3, 2012 at 15:02

entropy · Accepted Answer · 2012-07-03 15:11:22Z

3

Apparently, as of bash 3.2 the regular expression should not be quoted. So this should work:

#!/bin/bash
while read LINE
do
    if [[ $LINE =~ ^$ || $LINE =~ ^#.* ]] ; then
        continue;
    fi
    set -- $LINE
    local field1=$1
    local field2=$2
done < test.file

Edit: you should probably use Jo So's answer as it's definitely cleaner. But I was explaining why the regex fails and the reason behind the different behavior between OpenSuse and Ubuntu(different version of bash, very probably)

answered Jul 3, 2012 at 15:11

entropy

3,16423 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jo So Over a year ago

OMG, it is true! (Quoting man bash: Word splitting and pathname expansion are not performed on the words between the [[ and ]];) Bash is an ugly pile of crap, I would never use such ugly extensions!

Jo So · Accepted Answer · 2012-07-04 00:34:35Z

3

Quoting is wrong, that probably accounts for the regex failing.
No need to use bashisms.
No need to use set

Try

while read field1 field2 dummy
do
    if ! test "${field1%%#*}"
    then
        continue
    fi
    # do stuff here
done

EDIT: The obvious version using set

while read -r line
do
    if ! test "${line%%#*}"
    then
        continue
    fi
    set -- $line
    do_stuff_with "$@"
done

edited Jul 4, 2012 at 0:34

answered Jul 3, 2012 at 15:10

Jo So

26.9k6 gold badges45 silver badges60 bronze badges

2 Comments

user265330 Over a year ago

Thank you. I have to admit I don't understand the %% syntax. Also, the reason I use set is because I don't have a fixed number of fields on a line; I know my post suggested I do, but I was just trying to simplify as must as possible.

Jo So Over a year ago

So XyProblem strikes back. Regarding the %% syntax (that means "remove largest suffix glob-matching #*"), it's POSIX as opposed to the nasty regex bashisms. Look for "Parameter Expansion"

Sergii Stotskyi · Accepted Answer · 2012-07-03 15:10:32Z

-2

On my ubuntu there is no expresion like "=~" for test command. Just use this one:

if [[ $LINE = "" || ${LINE:0:1} = "#" ]] ; then
    continue;
fi

answered Jul 3, 2012 at 15:10

Sergii Stotskyi

5,4201 gold badge26 silver badges21 bronze badges

1 Comment

Dennis Williamson Over a year ago

=~ is a Bash operator for regex matching. If you have Bash, then you have it. The test command (or the equivalent [) don't support it, but the double square bracket form does.

Collectives™ on Stack Overflow

Bash script with regex not behaving on Ubuntu

3 Answers 3

1 Comment

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related