When you create array, you give it one value ([]) which goes to index 0.
On the first loop around it, j is 0 and array[j] is an array and you can assign a value to whatever array[j][k] is.
On the second loop around it, j is 1 but array[j] is undefined and you can't assign a value to a property of undefined.
You need to create an array each time you go around the outer loop.
// var array= new Array([]);
// Don't mix array literal and Array constructor syntax.
// Only create one array here.
var array = [];
var rows=8;
var cols=4;
for (var j=0; j< 8; j++){
// Create the array to hold the second dimension here
array[j] = [];
for (var k=0; k<4; k++){
array[j][k]=55;
console.log(j, k)
}
}