It sounds like most of your cells are unoccupied.
You can dramatically reduce memory if you store only occupied cells; this is termed "sparse" and is straightforward using a simple run-length-encoding (RLE) scheme (the runs of occupied and unoccupied cells are length-encoded).
For example, you could have an array of block instances:
Block[] blocks; // block[0] might be Grass, block[4] might be Water etc
Imagine that each X row on the 3D map is an integer array:
int[][][] data = new int[MAX_Z][MAX_Y][];
data[4][4] = new int[]{ 3, 2, 1, 1, 2, 1, 2, 3, 2, 2, 1 }; // some test row
The first integer says how many cells to skip, then how many cells are occupied, then the integer indices of the blocks in them, and so on. So that test row is:
| (x=0)
3 | skip 3 (x=3)
2 | occupied 2
1 | block[1]
| (x=4)
1 | block[1]
| (x=5)
2 | skip 2 (x=7)
1 | occupied 1
2 | block[2]
| (x=8)
3 | skip 3 (x=11)
2 | occupied 2
2 | block[2]
| (x=12)
1 | block[1]
Its straightforward to iterate over, and not too complicated to insert into:
void debug_line(int z,int y) {
int[] row = data[z][y];
if(row == null) return; // nothing on that line
int x = 0;
for(int i=0; i<row.length; ) {
int skip = row[i++], occupied = row[i++];
for(int j=0; j<skip; x++, j++) System.out.print("-");
for(int j=0; j<occupied; x++, j++) System.out.print(row[i++]);
}
}
This debug function iterates over all x in that row and emits this:
---11--2---21
You can likely use shorts or pack into bytes to dramatically reduce RAM further. And completely empty rows, rather than having an empty array (allocated and taking up space), can simply be NULL.
(I picked X rows rather than Z because its likely that you are drawing on the XY plane; you always want to be skipping along on the ground plane)
You can also likely share actual block instances; you don't need 100 instances of the 'grass' block if the properties are the same.
This will allow you to use 3D again.
More complicated structures like sparse octrees can reduce memory requirements much further. I'd recommend simple RLE first and then something 'better' if you hit performance or memory problems.