Understanding the Java stack

My bet is that by invoking print in the catch block you force another StackOverflowError that is caught by the outer block. Some of these calls will not have enough stack for actually writing the output stream.

The JLS says that:

Note that StackOverflowError, may be thrown synchronously by method invocation as well as asynchronously due to native method execution or Java virtual machine resource limitations.

The Java SE platform permits a small but bounded amount of execution to occur before an asynchronous exception is thrown.

The delay noted above is permitted to allow optimized code to detect and throw these exceptions at points where it is practical to handle them while obeying the semantics of the Java programming language. A simple implementation might poll for asynchronous exceptions at the point of each control transfer instruction. Since a program has a finite size, this provides a bound on the total delay in detecting an asynchronous exception.

The first time the StackOverFlowError occurs, the call to the last doAnything() is cancelled and the control is returned to the catch block from the last doAnything().

However, because the stack is still practically full, the simple fact of calling System.out.print("y") will causes another StackOverflowError because of the need of pushing some value on the stack and then make a call to the function print().

Therefore, another StackOverflowError occurs again and the return is now returned on the catch{} block of the previous doAnything(); where another StackOverflowError will happens because the need of stack space required to do a single call to System.out.println("y") is greater than the amount of space liberated from returning a call from doAnything().

Only when there will be enough space on the stack to execute a call to System.out.print("y") that this process will stop and a catch block will successfully complete. We can see that by running the following piece of code:

public class Principal3b {

    static int a = 0;
    static int i = 0;
    static int j = 0;

    public static void main(String[] args) {
      System.out.println("X");
        doAnything();
      System.out.println("Y");
        System.out.println(i);        
        System.out.println(j);
    }

    private static void doAnything() {

        a++;
        int b = a;

        try {
            doAnything();
        } catch (final Error e) {
            i++;
            System.out.println(a);
            System.out.println(b);
            j++;
        }
    }
}

Notice that a println(a) is used instead of a print(a); therefore a new line should be printed after each value of a if everything runs OK.

However, when I run it, I get the following result:

X
62066206620662066206620662066206
6190
Y
17
1

This means that there have been 17 attempts ro run the catch block. Of these catch block executions, 9 are unable to print anything before generating themselves a StackOverflowError; 7 are able to print the value of 6190 but are unable to print a newline after it before themselves rising an error again and finally, there is one that is able to both print the value of 6190 and the newline after it; therefore finally permitting its catch block to complete without any new StackOverflowError and return gracefully up the calls stack.

As we are dealing with StackOverflowError, these numbers are only an example and will vary greatly not only between machines but also between executions and the simple fact of adding or removing any kind of instructions should also change these values. However, the pattern seen here should remains the same.

One thing is clear that System.out.print("y"); in catch creates this puzzle. If we change the code as

static int n;

public static void main(final String[] args) throws Exception {
    System.out.println("1");
    doAnything();
    System.out.println(n);
}

private static void doAnything() {
    try {
        doAnything();
    } catch (Error e) {
        n++;
    }
}

it prints

1
1

Well the no. of times the stack overflow error is hit is undefined. However, the JVM allows you to recover from StackOverflowError error and continue execution of the system normally.

It is proved by the following code:

public class Really {
   public static void main(final String[] args) throws Exception {
     System.out.print("1");
     doAnything();
     System.out.println("2");
   }
   private static void doAnything() {
    try {
           throw new StackOverflowError();
       //doAnything();
    }
    catch(final Error e){
        System.out.print("y");
    }
   }
}

Note however, as @Javier said, the StackOverflowError is thrown by the JVM synchronously or asynchronously(which means it can be thrown by another thread, possibly a native thread) which is why it is not possible to get the stack trace of the error. The no. of times the thread(s) hit the catch() block is undefined.

In addition, Objects of type Error are not Exceptions objects they represent exceptional conditions. Errors represent unusual situations that are not caused by program errors, usually does not normally happen during program execution, such as JVM running out of memory. Although they share a common superclass Throwable, meaning both can be thrown, it can be placed in a catch but generally not supposed to be caught, as they represent rare, difficult-to-handle exceptional conditions.

Stack overflow.

You are only printing on exception,in the meantime the program recurses into overflow.

At which point this occurs depends on individual systems, memory, etc.

What is the purpose of the program?