Most of them only mentioned that the object header would occupy space, but their answers didn't answer my question, such as why the size of the object header affects the size of the array. The object header and the contents of the array are not stored in the same array. I mean that the object header and the array should not affect each other in terms of space occupation. Even if their combined size exceeds Interger.MAX_VALUE, their contents are not stored in the same array.
3 Answers
It's an interesting question. Using the Azul JDK on Windows, this:
final int[] arr = new int[Integer.MAX_VALUE-2];
gives rise to this error:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
whereas this:
final int[] arr = new int[Integer.MAX_VALUE-1];
elicits this error:
Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
which suggests that Integer.MAX_VALUE-2 is the maximum allowed array size on this particular JVM implementation. If people using other JVMs are getting a different result, then this may be implementation-specific.
Note, this answer suggests the value became Integer.MAX_VALUE-2 with Java 8, however the JDK ArraysSupport class contains this:
/**
* A soft maximum array length imposed by array growth computations.
* Some JVMs (such as HotSpot) have an implementation limit that will cause
*
* OutOfMemoryError("Requested array size exceeds VM limit")
*
* to be thrown if a request is made to allocate an array of some length near
* Integer.MAX_VALUE, even if there is sufficient heap available. The actual
* limit might depend on some JVM implementation-specific characteristics such
* as the object header size. The soft maximum value is chosen conservatively so
* as to be smaller than any implementation limit that is likely to be encountered.
*/
public static final int SOFT_MAX_ARRAY_LENGTH = Integer.MAX_VALUE - 8;
which suggests the actual value could be implementation-specfic and that Integer.MAX_VALUE-8 is being used as a conservative value (i.e. one that will generally work on most, if not all, JVMs).
If we use the JOL library to see the class layout:
System.out.println(ClassLayout.parseClass(int[].class).toPrintable());
it gives us this:
OFF SZ TYPE DESCRIPTION VALUE
0 8 (object header: mark) N/A
8 4 (object header: class) N/A
12 4 (array length) N/A
16 0 int [I.<elements> N/A
Instance size: 16 bytes
I.e. the layout in memory consists of 16 bytes of metadata, followed by the actual array data itself.
The above results were obtained using Java 21, however I get the exact same results for Java 11.
This leads us to the two following conclusions:
- The maximum allowed array size is typically Integer.MAX_VALUE-2 (Java 8 onwards) but is JVM-specific. Integer.MAX_VALUE-8 is used as a conservative soft limit that will work on most JVMs.
- The number of bytes required for the array metadata doesn't appear to be a factor in the maximum allowed array size.
Comments
Your assertion of "object header and the contents of the array are not stored in the same array" is not correct. Looking at the linked article of the answer of your first linked question (archive link), an Array object stores all of its data in a single contiguous region of memory, starting with the object header and the length.
See the image from the article:
Note: If the array had more than a single int element, it would just get appended on.
How the memory is laid out within the JVM has not much to do with how you access it from code. If you, for example, access arrObj[2] the JVM will "calculcate" the offset for you to get the right element.
This answer also provides some reasoning why a Java Object generally shouldn't exceed a size of Integer.MAX_VALUE
2 Comments
exception
My guess is that Java array objects are mapped to C++ arrays within the JVM, and the upper limit of C++ arrays is Integer.MAX_VALUE - 1. Further, when mapping Java arrays to C++, does it include some part of an object header? And this part is stored within a C++ array. I don't have a deep understanding of C++, so I'm not sure if my interpretation is correct. If that's the case, why can't we separate the array data and object header in C++ into different arrays?
JensV
@exception while I don't know much of the internals myself (the other answers highlight some of those details) I can give you some informed guesses why it is that way. If you access or modify an array, the JVM has to access the size (and possibly flags) anyways to perform tasks like bounds-checking. Having the data next to each other may increase performance when the cpu fetches lines of memory. While you could store it separately, there's not much practical use, since why would you ever create an array of that size anyways? Simpler and consistent is usually better than catering for edge cases
There is a hard limit established in arrayOop.hpp:
// It should be ok to return max_jint here, but parts of the code // (CollectedHeap, Klass::oop_oop_iterate(), and more) uses an int for // passing around the size (in words) of an object. So, we need to avoid // overflowing an int when we add the header. See CRs 4718400 and 7110613.
This hard limit depends on the JVM version and factors like 32 bit / 64 bit architecture. For a 64 bit JVM from Java 8 onwards this seems to be Integer.MAX_VALUE-2.
There is a soft limit used in the Java standard classes (collections, buffers etc) that automatically expand when more space is needed. These classes use a soft limit of Integer.MAX_VALUE-8 - probably to guarantee the same resize behaviour independent of the JVM.
This are upper bounds that are further restricted by available heap. You cannot allocate a new int[Integer.MAX_VALUE-2] if your maximum heap size is 100 MB.
ArrayList. With Oracle 1.8.0 and with OpenJDK 23 I can create anarraywithInteger.MAX_VALUE-2...