Structured numpy array with 2 different data types

Question

I imported a csv file into a numpy array which I need to convert to a structured array with only the first column as dtype string and all the other 47 columns as float. How do I define data type for the other 47 columns in a single operation? Do I have to specify dtype column by column?

Thanks in advance

Do you need numpy array for all the table? Would storing each column in separate table be fine (then possibly creating a list from the column arrays)? — Jan Stránský
– Jan Stránský, Commented Aug 28, 2020 at 7:15
@JanStránský thank you for your reply. Yes, I need all the columns on the array. the first column is an identifier with duplicates and the other 47 columns are float numbers that I need to run complex calculations — Neyls
– Neyls, Commented Aug 28, 2020 at 7:22
Have you considered using pandas for storing the mixed data types data? — FBruzzesi
– FBruzzesi, Commented Aug 28, 2020 at 7:24
@FBruzzesi - I have to use numpy unless I can run a deap evolutionary algorithm using Pandas — Neyls
– Neyls, Commented Aug 28, 2020 at 7:29

Valdi_Bo · Accepted Answer · 2020-08-28 09:46:00Z

You can read the source file just as a structured array.

Assume that you input file contains:

one string field, to be named as Id,
just four float fields, to be named as F1, F2 and so on.

So its content is:

ABCD,160.72,180.21,260.13,451.48
EFGH,252.42,132.21,150.11,612.56
IJKL,541.77,455.21,268.76,543.81

To read such a file you can use np.loadtxt method, passing dtype as a structured type (a list of definitions), which can be generated e.g. in a list comprehension:

nFloats = 4
a = np.loadtxt('Input.csv', delimiter=',',
    dtype=[('Id', 'U10')] + [( f'F{i+1}', '<f4' ) for i in range(nFloats)])

Note that I passed U10 as the type of Id column (10 chars). If you need, set other size of this field.

The result is:

array([('ABCD', 160.72, 180.21, 260.13, 451.48),
       ('EFGH', 252.42, 132.21, 150.11, 612.56),
       ('IJKL', 541.77, 455.21, 268.76, 543.81)],
      dtype=[('Id', '<U10'), ('F1', '<f4'), ('F2', '<f4'), ('F3', '<f4'), ('F4', '<f4')])

Of course, in your target version of code increase nFloats accordingly (probably it should be 47).

Collectives™ on Stack Overflow

Structured numpy array with 2 different data types

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related