testing a program in bash

Question

I wrote a program in c++ and now I have a binary. I have also generated a bunch of tests for testing. Now I want to automate the process of testing with bash. I want to save three things in one execution of my binary:

execution time
exit code
output of the program

Right now I am stack up with a script that only tests that binary does its job and returns 0 and doesn't save any information that I mentioned above. My script looks like this

#!/bin/bash

if [ "$#" -ne 2 ]; then
    echo "Usage: testScript <binary> <dir_with_tests>"
    exit 1
fi

binary="$1"
testsDir="$2"

for test in $(find $testsDir -name '*.txt'); do
    testname=$(basename $test)

    encodedTmp=$(mktemp /tmp/encoded_$testname)
    decodedTmp=$(mktemp /tmp/decoded_$testname)

    printf 'testing on %s...\n' "$testname"

    if ! "$binary" -c -f $test -o $encodedTmp > /dev/null; then
        echo 'encoder failed'
        rm "$encodedTmp"
        rm "$decodedTmp"
        continue
    fi

    if ! "$binary" -u -f $encodedTmp -o $decodedTmp > /dev/null; then
        echo 'decoder failed'
        rm "$encodedTmp"
        rm "$decodedTmp"
        continue
    fi

    if ! diff "$test" "$decodedTmp" > /dev/null ; then
        echo "result differs with input"
    else
        echo "$testname passed"
    fi

    rm "$encodedTmp"
    rm "$decodedTmp"
done

I want save output of $binary in a variable and not send it into /dev/null. I also want to save time using time bash function

Anthony Geoghegan · Accepted Answer · 2015-10-09 11:35:14Z

1

As you asked for the output to be saved in a shell variable, I tried answering this without using output redirection – which saves output in (temporary) text files (which then have to be cleaned).

Saving the command output

You can replace this line

if ! "$binary" -c -f $test -o $encodedTmp > /dev/null; then

with

if ! output=$("$binary" -c -f $test -o $encodedTmp); then

Using command substitution saves the program output of $binary in the shell variable. Command substitution (combined with shell variable assignment) also allows exit codes of programs to be passed up to the calling shell so the conditional if statement will continue to check if $binary executed without error.

You can view the program output by running echo "$output".

Saving the time

Without a more sophisticated form of Inter-Process Communication, there’s no way for a shell that’s a sub-process of another shell to change the variables or the environment of its parent process so the only way that I could save both the time and the program output was to combine them in the one variable:

if ! time-output=$(time "$binary" -c -f $test -o $encodedTmp) 2>&1); then

Since time prints its profiling information to stderr, I use the parentheses operator to run the command in subshell whose stderr can be redirected to stdout. The programming output and the output of time can be viewed by running echo "$time-output" which should return something similar to:

<program output>
<blank line>
real    0m0.041s
user    0m0.000s
sys     0m0.046s

answered Oct 9, 2015 at 11:35

Anthony Geoghegan

12.1k5 gold badges54 silver badges59 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

antonpp Over a year ago

Thx, that is almost exactly what I wanted. It would be even better if I could have output of the program and time info in separate files/variables. But anyway, I can handle it like this.

Anthony Geoghegan Over a year ago

@Anton A while back, I spent a long time trying to do something similar but couldn't figure out any other way to do it without creating temporary files. Glad to be able to help.

Szymon Roziewski · Accepted Answer · 2015-10-09 11:14:15Z

1

You can get the process status in bash by using $? and print it out by echo $?. And to catch the output of time, you could use sth like that

{ time sleep 1 ; } 2> time.txt

Or you can save the output of the program and execution time at once

(time ls) > out.file 2>&1

answered Oct 9, 2015 at 11:14

Szymon Roziewski

1,0312 gold badges29 silver badges40 bronze badges

Comments

Kadir · Accepted Answer · 2015-10-09 11:19:58Z

1

You can save output to a file using output redirection. Just change first /dev/null line:

if ! "$binary" -c -f $test -o $encodedTmp > /dev/null; then

to

if ! "$binary" -c -f $test -o $encodedTmp > prog_output; then

then change second and third /dev/null lines respectively:

if ! "$binary" -u -f $encodedTmp -o $decodedTmp >> prog_output; then

if ! diff "$test" "$decodedTmp" >> prog_output; then

To measure program execution put

start=$(date +%s)

on the first line

then

end=$(date +%s)
echo "Execution time in seconds: " $((end-start)) >> prog_output

on the end.

answered Oct 9, 2015 at 11:19

Kadir

1,7822 gold badges20 silver badges24 bronze badges

2 Comments

Vadim Landa Over a year ago

Alternatively, you can use the time command to measure execution duration.

Kadir Over a year ago

time utility is to summarize system resource usage.

Collectives™ on Stack Overflow

testing a program in bash

3 Answers 3

Saving the command output

Saving the time

2 Comments

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Saving the command output

Saving the time

2 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related