How to create a numpy array of views to other numpy arrays

Question

I have a large list of objects that include numpy arrays as attributes. They each have methods that manipulate the array. I would like to create a single 2D numpy array that stores the other arrays and updates when the individual numpy arrays are manipulated.

This is easy to accomplish with lists as you simply need to create a list of references to other lists.

>>> x = [1,2,3]
>>> y = [4,5,6]
>>> z = [x,y] # stores reference to x and y
>>> x[0] = 10
>>> z
[[10,2,3],[4,5,6]]

However, doing the same in numpy creates copies of the object.

>>> x = np.array([1,2,3])
>>> y = np.array([4,5,6])
>>> z = np.array([x,y])  # setting the optional argument copy = False didn't help either
>>> id(x)
140673084678272
>>> id(z[0])
140673084678512

I guessed that setting copy = False wouldn't work because I'm passing a new concatenated list object which hadn't existed before. Is there a way to create z where it's elements are references to the numpy arrays x and y?

I recognize that references in numpy are typically accomplished with views, but this seems to be a different use case. Creating a view object of a single numpy array is fairly straightforward, but I'm unsure how to store in a numpy array, N view objects from N individual numpy arrays.

The method which manipulates the arrays requires a numpy array, and the final method I'l be using on the concatenated array requires a numpy array. — Adam Birenbaum
– Adam Birenbaum, Commented Jan 22, 2020 at 17:22
You could make the new array, and then 'views' of the individual rows (discarding or ignoring the originals). — hpaulj
– hpaulj, Commented Jan 22, 2020 at 18:16
A view only works with the array can share the underlying databuffer - that is access the same data as the original, but with different shape and strides. Often that is a subset, but never any sort of superset of several databuffers. — hpaulj
– hpaulj, Commented Jan 22, 2020 at 18:17

Dave Rove · Accepted Answer · 2022-06-14 09:13:09Z

1

Create an empty object array of the required length (it contains Nulls). Assign the list to a view of the whole array. Done. And yes, the original objects update when the individual numpy arrays are manipulated. Note that operations on the whole array, however, always seem to create copies even when you'd normally expect in-place operations.

import numpy as np
x = np.array([1,2,3])
y = np.array([4,5,6])
z = np.empty(2, dtype=object)
z[:] = [x,y]
print(id(x))
print(id(z[0]))

Output:

140293181519152
140293181519152

answered Jun 14, 2022 at 9:13

Dave Rove

9834 gold badges17 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Poe Dator · Accepted Answer · 2020-01-22 17:40:26Z

0

One solution would be to stack() or concatenate() arrays i certain dimensions match. if dims don't match - use padding, store dimensions in separate list and then stack.

answered Jan 22, 2020 at 17:40

Poe Dator

4,8522 gold badges20 silver badges43 bronze badges

2 Comments

Adam Birenbaum Over a year ago

Those methods do combine the arrays in the way I'm looking for, but after some testing, it seems they do it by making copies, i.e. after combining them, changing the individual arrays doesn't affect the combined array. I'm wondering if there exists a stack or concatenate which combines the references or view objects rather than a new copy of the array.

hpaulj Over a year ago

There is numpy object dtype, but I view that as a bastardized list. It is created most often by mistake as in np.array([[1,2,3], [4,5]]). Math on it hit-or-miss and slower than on a regular numeric array, and iteration is slower than on a list.

Collectives™ on Stack Overflow

How to create a numpy array of views to other numpy arrays

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related