Shell-script does not give expected output in Ubuntu

Question

Shell Script:

#!/bin/sh
# -*-sh-*-
java -classpath Test.jar Test test1.xml > javaOutput 2>&1;
if cat javaOutput | tr '\n' ' ' | grep ".*java.lang.IndexOutOfBoundsException0.*ArrayList.java:653.*Test.java:142.*" &>/dev/null; then
    echo TRUE;
else
    echo FALSE;
fi

Output File Content (javaOutput):

0,2,468.000000
1,2,305.000000
2,5,2702.000000
3,3,1672.000000
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
    at java.util.ArrayList.rangeCheck(ArrayList.java:653)
    at java.util.ArrayList.get(ArrayList.java:429)
    at Test.processPayments(Test.java:113)
    at Test.processFile(Test.java:131)
    at Test.main(Test.java:142)

I am using the following version of ubuntu:

Distributor ID: Ubuntu
Description:    Ubuntu 17.04
Release:    17.04

When I copy and paste the script in the command prompt it works fine and echos FALSE as expected but whenever I am executing the script it keeps echoing TRUE. I even ran this script in mac os and it executes as expected in MacOs. I am very puzzled. Any help or insight would be appreciated.

Note: The expectation here is to get FALSE since there is no string IndexOutOfBoundsException0 in the output content.

If you remove &>/dev/null from the script, what does it print? Also, recent versions of Ubuntu use dash as /bin/sh (instead of the more common bash). I can't see anything dash would treat differently than bash, but try setting the shebang to #!/bin/bash and see if switches behavior. — Gordon Davisson
– Gordon Davisson, Commented Apr 8, 2018 at 6:33
@GordonDavisson good call. I removed &>/dev/null and it started working, however for the cases which it echos TRUE it is now printing the output of the grep. The requirement for the script is to not generate the output when its evaluated to true. Do you know if there is any other way to achieve it? — sophist_pt
– sophist_pt, Commented Apr 8, 2018 at 6:41
Another question that might be of interest: Is there any sh code that is not valid bash code? — Wildcard
– Wildcard, Commented Jun 14, 2019 at 20:08

Gordon Davisson · Accepted Answer · 2018-04-08 06:55:43Z

4

I figured it out; it's a small syntax difference between bash (a shell program with a lot of features added on top of the basic set specified by the POSIX standard) and dash (a bare-minimum POSIX shell). Many OSes use bash as /bin/sh, but Debian (and Ubuntu) switched to dash a while back.

The syntax difference is that the shorthand &> to redirect both standard output and standard error to the same place is a bash extension; dash will parse this as two separate tokens, a & (a command delimiter which says to run the command in the background), and > (a redirection of standard out). Since & marks the end of one command and the beginning of another, the redirection doesn't apply to the preceding command, it's sort of a minimal command of its own. Essentially, dash is parsing the thing between if and then as:

cat javaOutput | tr '\n' ' ' | grep ".*java.lang.IndexOutOfBoundsException0.*ArrayList.java:653.*Test.java:142.*" &
>/dev/null

...and if only looks at the success/failure of the last command in that block, which is just a redirect... which always succeeds.

Fortunately, the solution is simple: don't use the bash shorthand, use the full 2>&1 to redirect both stdout and stderr:

if cat javaOutput | tr '\n' ' ' | grep ".*java.lang.IndexOutOfBoundsException0.*ArrayList.java:653.*Test.java:142.*" >/dev/null 2>&1; then

answered Apr 8, 2018 at 6:55

Gordon Davisson

128k16 gold badges141 silver badges168 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

sophist_pt Over a year ago

Thanks a lot. It worked perfectly. Good to know the actual root cause.

Collectives™ on Stack Overflow

Shell-script does not give expected output in Ubuntu

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related