Python Numpy append array without flattening

Question

In Python 3 I am importing several data files in a loop, and I would like to be able to store all the data in a single 2-dimensional array. I start with something like data = np.array([]) and on each iteration i want to add a new array datai = np.array([1,2,3]), how can I get my final array to look like this? [[1,2,3],[1,2,3],...,[1,2,3]]

I have tried np.append, np.concatenate, and np.stack, but none seem to work. An example code that I'm trying:

data = np.array([])
for i in range(datalen):
    datai = *func to load data as array*
    data = np.append(data, datai)

but of course this returns a flattened array. Is there any way I can get back a 2-dimensional array of length datalen with each element being the array datai?

Thanks!

Looks like you want vstack. Are all of your subarrays the same length? — user3483203
– user3483203, Commented Aug 3, 2018 at 16:11
yes! Ok i can get that to work for all iterations after the first, but since I start with an empty array it gives me a dimension error. I can just throw in an if statement to save the first array and vstack from then forward, but is there any cleaner way to do it? — Vicente Esnaola
– Vicente Esnaola, Commented Aug 3, 2018 at 16:26
Look at the np.append docs. What does it say about omitting the axis parameter? — hpaulj
– hpaulj, Commented Aug 3, 2018 at 16:28
You are trying to use a list append model for arrays. np.empty([]) is not the same as the list []. Look at its shape and ndim. For concatenate to work, the inputs have to have matching dimensions. data1 is 1d (3,). It can be joined on axis 0 to another 1d array. To join on a new initial axis it needs to be 2d (which is what vstack addresses). In any case repeated list append is the right way, not a repeated concatenate. — hpaulj
– hpaulj, Commented Aug 3, 2018 at 16:45
What's the shape of data1? How do you load it? Often, when loaded from a txt file, the resulting array will be 2d. It could of course be 1d. — hpaulj
– hpaulj, Commented Aug 3, 2018 at 17:20

justengel · Accepted Answer · 2018-08-03 16:42:06Z

5

The fastest way would be vstack

data = np.vstack((get_data() for i in range(datalen)))

vstack requires a tuple/iterable

data = np.vstack((data1, data2, data3))

or you can do this by appending with axis=0

data = np.empty(shape=(0, 3))
data = np.append(data, datai.reshape((-1, 3)), axis=0)  # -1 will make the rows automatic

answered Aug 3, 2018 at 16:42

justengel

6,3704 gold badges28 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Khaled Khazem · Accepted Answer · 2021-01-27 18:39:01Z

1

You can reshape your array using np.reshape like this

flattened_array = np.array([1,2,3,1,2,3,1,2,3])

wanted_array = np.reshape(flattened_array, (-1, 3))

This would result in

[[1, 2, 3],[1, 2, 3],[1, 2, 3]]

answered Jan 27, 2021 at 18:39

Khaled Khazem

233 bronze badges

1 Comment

Travelling Salesman Over a year ago

my way to solve it! It seems dirty though, no?

Sheldore · Accepted Answer · 2018-08-03 16:30:18Z

0

Solution 1 using list comprehensions:

data = []
datalen = 4
datai = range(1,4)
data = [list(datai) for _ in range(datalen)]
print (data)

Output

[[1, 2, 3], [1, 2, 3], [1, 2, 3], [1, 2, 3]]

Solution 2 (just a bit lengthy)

data = []
datalen = 4

for i in range(datalen):
    datai = range(1,4)
    data.append(list(datai))
print (data)

with the same output as above. In the second method you can also just simply use data.append(list(range(1,4))). You can choose if you want to convert datai to list or not. If you want the final output as an array, you can just use np.array()

edited Aug 3, 2018 at 16:30

answered Aug 3, 2018 at 16:25

Sheldore

39.2k9 gold badges63 silver badges76 bronze badges

2 Comments

kevinkayaks Over a year ago

method 2: why call np.arange(1,4) instead of range(1,4) when you're going to convert it to list anyway?

Sheldore Over a year ago

Good catch. Fixed! Thanks

Kaustumbh Jaiswal · Accepted Answer · 2018-08-03 16:37:26Z

0

You can try this-

data = np.zeros(shape=(datalen,len(datai))
for i in range(datalen):
   data[i] = datai

answered Aug 3, 2018 at 16:37

Kaustumbh Jaiswal

11 bronze badge

1 Comment

Kaustumbh Jaiswal Over a year ago

If you wish to do it using NumPy.

Ufos · Accepted Answer · 2020-06-25 23:43:01Z

0

It's called numpy.tile.

From the official docs:

>>> c = np.array([1,2,3,4])
>>> np.tile(c,(3,1))
array([[1, 2, 3, 4],
       [1, 2, 3, 4],
       [1, 2, 3, 4]])

so, for your datai do np.tile(datai,(N_repeats,1))

answered Jun 25, 2020 at 23:43

Ufos

3,3162 gold badges35 silver badges39 bronze badges

Collectives™ on Stack Overflow

Python Numpy append array without flattening

5 Answers 5

Comments

1 Comment

2 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

1 Comment

2 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related