Numpy reshape list to 3D Array

Question

I have a list of arrays that I would like to be reshaped. Each array is a trial, the columns within each array is a feature, and the rows in each array is the timestep. I would like the list reshaped to (trial, timestep, feature). As an example, D is what I am trying to convert to a 3D array - the timesteps are not uniform.

A = np.random.rand(3,10) #Trial 1 has 3 timesteps and ten features
B = np.random.rand(10,10) #Trial 2 has 10 timesteps and ten features
C = np.random.rand(7,10) #Trial 3 has 7 timesteps and ten features
D = [A,B,C,D] #Data as given in the form of a list

How am I able to get a 3d array with variable timesteps? I am trying to use this an input to a keras neural network

What is application? are you allowed to interpolate between timesteps to make all trials even? — sai
– sai, Commented Dec 9, 2020 at 21:16
@sai It is for a sequence classification application. Based on what QuangHoang said, I will just pad the arrays then do the reshape — GK89
– GK89, Commented Dec 9, 2020 at 21:23
Might not be the best idea, but feel free to try. I would've however tried to downsample all the trials to 3 timesteps or upsample the less frequent trials if I have an idea of what the features are and how they might vary based on their physics — sai
– sai, Commented Dec 9, 2020 at 21:27

Pedro Marques · Accepted Answer · 2020-12-09 21:40:48Z

1

You can do the following:

D = tf.ragged.stack([tf.RaggedTensor.from_tensor(x) for x in [A, B, C]])

This yields an ragged tensor with shape: TensorShape([3, None, None])

Or

values = np.vstack([A, B, C])
D = tf.RaggedTensor.from_row_lengths(values, [x.shape[0] for x in [A, B, C]])

which yields a ragged tensor with shape: (3, None, 10)

Working with ragged tensors in keras can be tricky.

Typically, for most applications the best choice is to use a reasonable number for the maximum sequence dimension and mask the sequences that are empty. For some applications I'm working on that is not a very attractive option because I have lots of documents with very small sequences and then some with very large sequences. But if you don't feel really comfortable with keras/tensorflow mechanics you should probably avoid using ragged tensors.

answered Dec 9, 2020 at 21:40

Pedro Marques

2,6821 gold badge13 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Numpy reshape list to 3D Array

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related