I know nothing about Ant debugging, but there are two approaches to doing what you want to do in Bash. The first is to test output like you've shown:
if test $(ant debug) == 'SUCCESS'; then
# do stuff
else
# do other stuff
fi
You can make your shell script portable to other variants on the Bourne shell by using backticks instead of $(.....) like you wrote in your question, but that starts to become a hassle if your commands later involve nested quotes.
The second way, which is a little more robust, is to test the exit value of the commands instead of their output. This depends on Ant (or whatever) having exit codes that are documented and stable, but it means that if the output of the commands changes, your scripts will continue to work. For example, the POSIX standard says that if a programs succeeds in doing whatever it's supposed to do, it should exit() with a value of zero:
ant debug > /dev/null
ant_exit_code=$?
# other commands can safely go here now that $? is captured
if test $ant_exit_code -eq 0; then
# do stuff
else
# do other stuff
fi
And yes, Bourne shell really does end an if block with "fi". :-)