How do you create a structured array from two columns in a DataFrame? I tried this:
df = pd.DataFrame(data=[[1,2],[10,20]], columns=['a','b'])
df
a b
0 1 2
1 10 20
x = np.array([([val for val in list(df['a'])],
[val for val in list(df['b'])])])
But this gives me this:
array([[[ 1, 10],
[ 2, 20]]])
But I wanted this:
[(1,2),(10,20)]
Thanks!
There are a couple of methods. You may experience a loss in performance and functionality relative to regular NumPy arrays.
You can use pd.DataFrame.to_records with index=False. Technically, this is a record array, but for many purposes this will be sufficient.
res1 = df.to_records(index=False)
print(res1)
rec.array([(1, 2), (10, 20)],
dtype=[('a', '<i8'), ('b', '<i8')])
Manually, you can construct a structured array via conversion to tuple by row, then specifying a list of tuples for the dtype parameter.
s = df.dtypes
res2 = np.array([tuple(x) for x in df.values], dtype=list(zip(s.index, s)))
print(res2)
array([(1, 2), (10, 20)],
dtype=[('a', '<i8'), ('b', '<i8')])
What's the difference?
Very little. recarray is a subclass of ndarray, the regular NumPy array type. On the other hand, the structured array in the second example is of type ndarray.
type(res1) # numpy.recarray
isinstance(res1, np.ndarray) # True
type(res2) # numpy.ndarray
The main difference is record arrays facilitate attribute lookup, while structured arrays will yield AttributeError:
print(res1.a)
array([ 1, 10], dtype=int64)
print(res2.a)
AttributeError: 'numpy.ndarray' object has no attribute 'a'
Related: NumPy “record array” or “structured array” or “recarray”
Use list comprehension for convert nested lists to tuples:
print ([tuple(x) for x in df.values.tolist()])
[(1, 2), (10, 20)]
Detail:
print (df.values.tolist())
[[1, 2], [10, 20]]
EDIT: You can convert by to_records and then to np.asarray, check link:
df = pd.DataFrame(data=[[True, 1,2],[False, 10,20]], columns=['a','b','c'])
print (df)
a b c
0 True 1 2
1 False 10 20
print (np.asarray(df.to_records(index=False)))
[( True, 1, 2) (False, 10, 20)]
Here's a one-liner:
list(df.apply(lambda x: tuple(x), axis=1))
or
df.apply(lambda x: tuple(x), axis=1).values
object dtype for integers, even with structured arrays.