numpy Stacking 1D arrays into structured array

Question

I'm running Numpy 1.6 in Python 2.7, and have some 1D arrays I'm getting from another module. I would like to take these arrays and pack them into a structured array so I can index the original 1D arrays by name. I am having trouble figuring out how to get the 1D arrays into a 2D array and make the dtype access the right data. My MWE is as follows:

>>> import numpy as np
>>> 
>>> x = np.random.randint(10,size=3)
>>> y = np.random.randint(10,size=3)
>>> z = np.random.randint(10,size=3)
>>> x
array([9, 4, 7])
>>> y
array([5, 8, 0])
>>> z
array([2, 3, 6])
>>> 
>>> w = np.array([x,y,z])
>>> w.dtype=[('x','i4'),('y','i4'),('z','i4')]
>>> w
array([[(9, 4, 7)],
       [(5, 8, 0)],
       [(2, 3, 6)]], 
      dtype=[('x', '<i4'), ('y', '<i4'), ('z', '<i4')])
>>> w['x']
array([[9],
       [5],
       [2]])
>>> 
>>> u = np.vstack((x,y,z))
>>> u.dtype=[('x','i4'),('y','i4'),('z','i4')]
>>> u
array([[(9, 4, 7)],
       [(5, 8, 0)],
       [(2, 3, 6)]],    
      dtype=[('x', '<i4'), ('y', '<i4'), ('z', '<i4')]) 

>>> u['x']
array([[9],
       [5],
       [2]])

>>> v = np.column_stack((x,y,z))
>>> v
array([[(9, 4, 7), (5, 8, 0), (2, 3, 6)]], 
      dtype=[('x', '<i4'), ('y', '<i4'), ('z', '<i4')])

>>> v.dtype=[('x','i4'),('y','i4'),('z','i4')]
>>> v['x']
array([[9, 5, 2]])

As you can see, while my original x array contains [9,4,7], no way I've attempted to stack the arrays and then index by 'x' returns the original x array. Is there a way to do this, or am I coming at it wrong?

Do you need to operate on the 2d array? Why not just use a dictionary? — OregonTrail
– OregonTrail, Commented Jul 3, 2013 at 19:06
I guess I just assumed it would be better to not mix data types and use ndarray since it supported dict-like indexing, but there's no real sound reasoning behind that. — Thav
– Thav, Commented Jul 3, 2013 at 20:11
To answer the first question, no, in this case I don't need to operate on the 2d array. — Thav
– Thav, Commented Jul 3, 2013 at 20:20

Mark · Accepted Answer · 2015-07-23 03:38:12Z

17

One way to go is

wtype=np.dtype([('x',x.dtype),('y',y.dtype),('z',z.dtype)])
w=np.empty(len(x),dtype=wtype)
w['x']=x
w['y']=y
w['z']=z

Notice that the size of each number returned by randint depends on your platform, so instead of an int32, i.e. 'i4', on my machine I have an int64 which is 'i8'. This other way is more portable.

edited Jul 23, 2015 at 3:38

Mark

3891 gold badge7 silver badges20 bronze badges

answered Jul 3, 2013 at 21:05

gg349

22.8k5 gold badges58 silver badges65 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jaime Over a year ago

+1 This is actually the only way to get arrays with different dtypes into a single structured array.

Stefano Over a year ago

Is the a way to do it when x, y, z, have different number of elements?

Jaime · Accepted Answer · 2013-07-03 21:16:13Z

3

You want to use np.column_stack:

import numpy as np

x = np.random.randint(10,size=3)
y = np.random.randint(10,size=3)
z = np.random.randint(10,size=3)

w = np.column_stack((x, y, z))
w = w.ravel().view([('x', x.dtype), ('y', y.dtype), ('z', z.dtype)])

>>> w
array([(5, 1, 8), (8, 4, 9), (4, 2, 6)], 
      dtype=[('x', '<i4'), ('y', '<i4'), ('z', '<i4')])
>>> x
array([5, 8, 4])
>>> y
array([1, 4, 2])
>>> z
array([8, 9, 6])
>>> w['x']
array([5, 8, 4])
>>> w['y']
array([1, 4, 2])
>>> w['z']
array([8, 9, 6])

answered Jul 3, 2013 at 21:16

Jaime

67.7k19 gold badges128 silver badges164 bronze badges

2 Comments

Thav Over a year ago

I used column_stack in my example, but didn't get the same results as you did. I would guess the difference is in the w.rave1()... line, but I don't quite understand what's happening there.

Brecht Machiels Over a year ago

This will fail if the datatypes of the 1D arrays occupy a different number of bytes.

Jordan Kowal · Accepted Answer · 2019-11-11 10:53:45Z

To build on top of the chosen answer, you can make this process dynamic:

You first loop over your arrays (which can be single columns)
Then you loop over your columns to get the datatypes
You create the empty array using those datatypes
Then we repeat those loops to populate the array

SETUP

# First, let's build a structured array
rows = [
    ("A", 1),
    ("B", 2),
    ("C", 3),
]
dtype = [
    ("letter", str, 1),
    ("number", int, 1),
]
arr = np.array(rows, dtype=dtype)

# Then, let's create a standalone column, of the same length:
rows = [
    1.0,
    2.0,
    3.0,
]
dtype = [
    ("float", float, 1)
]
new_col = np.array(rows, dtype=dtype)

SOLVING THE PROBLEM

# Now, we dynamically create an empty array with the dtypes from our structured array and our new column:
dtypes = []
for array in [arr, new_col]:
    for name in array.dtype.names:
        dtype = (name, array[name].dtype)
        dtypes.append(dtype)
new_arr = np.empty(len(new_col), dtype=dtypes)

# Finally, put your data in the empty array:
for array in [arr, new_col]:
    for name in array.dtype.names:
        new_arr[name] = array[name]

Hope it helps

leslem · Accepted Answer · 2013-07-03 22:07:35Z

0

You might want to look into numpy's record arrays for this use:

"Numpy provides powerful capabilities to create arrays of structs or records. These arrays permit one to manipulate the data by the structs or by fields of the struct."

Here's documentation on the record arrays: http://docs.scipy.org/doc/numpy/user/basics.rec.html

You can use your variable names as the field names.

answered Jul 3, 2013 at 22:07

leslem

731 silver badge4 bronze badges

Comments

OregonTrail · Accepted Answer · 2013-07-03 21:10:29Z

-2

Use a dictionary

#!/usr/bin/env python

import numpy

w = {}
for key in ('x', 'y', 'z'):
    w[key] = np.random.randint(10, size=3)

print w

answered Jul 3, 2013 at 21:10

OregonTrail

9,0897 gold badges46 silver badges63 bronze badges

Collectives™ on Stack Overflow

numpy Stacking 1D arrays into structured array

5 Answers 5

2 Comments

2 Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

2 Comments

2 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related