save a multidimensional array with different sub-array length python

Question

I tried to save a multidimensional array with different sub-array length into a text file by python. My code is:

import numpy as np
a = np.arange(10)
b = np.arange(5)*0.1
c = np.arange(3)*10
v = [list(a),list(b),list(c)]
v = np.array(v)
v = v.T
np.savetxt("file.dat",v)

The result shows:

TypeError: Mismatch between array dtype ('object') and format specifier ('%.18e')

I think that it is because the lengths of the sub-arrays(lists) are not same.

Would you please help me to solve this problem?

Thank you,

Isaac

The problem is that a, b, and c don't have the same len, so v is a "jagged" list, so when you call np.array(v) it's making an array of objects, each of those objects being a python list. Numpy doesn't support jagged arrays, really. Essentially, you can work with an array of lists, or an array of arrays, but the dtype has to be object, which pretty much defeats the purpose of numpy. Just use Python lists. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Jan 19, 2017 at 1:44
Thank you juanpa.arrivillaga: I would like to save a, b, c as columns in the same file. Would you please suggest any way to do it? v.T does not support list., — Isaac
– Isaac, Commented Jan 19, 2017 at 2:40

hpaulj · Accepted Answer · 2017-01-19 03:21:56Z

2

Look at v before the savetxt:

In [34]: v
Out[34]: 
array([[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
       [0.0, 0.10000000000000001, 0.20000000000000001, 0.30000000000000004, 0.40000000000000002],
       [0, 10, 20]], dtype=object)

In [36]: v.shape
Out[36]: (3,)

It's a 1d, 3 element array that contains your 3 arrays. .T does nothing. savetxt is designed to save a 2d table. You may need to experiment with creating arrays from components like this. The fact that they differ in length is important.

At best savetxt can write 3 lists to the file:

In [39]: np.savetxt('test.txt',v,fmt='%s')
In [40]: cat test.txt
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[0.0, 0.10000000000000001, 0.20000000000000001, 0.30000000000000004, 0.40000000000000002]
[0, 10, 20]

What output were you expecting?

==============

zip_longest can create 'columns'. The default fill is None. I can fill with an empty string, but then I have to use the %s fmt.

In [50]: for v in itertools.zip_longest(a,b,c,fillvalue=''):
    ...:     print('%5s, %5s. %5s'%tuple(v))

    0,   0.0.     0
    1,   0.1.    10
    2,   0.2.    20
    3,   0.3.      
    4,   0.4.      
    5,      .      
    6,      .      
    7,      .      
    8,      .      
    9,      .

Or with an intermediate array:

In [51]: arr=np.array(list(itertools.zip_longest(a,b,c,fillvalue='')))
In [52]: arr
Out[52]: 
array([['0', '0.0', '0'],
       ['1', '0.1', '10'],
       ['2', '0.2', '20'],
       ['3', '0.3', ''],
       ['4', '0.4', ''],
       ['5', '', ''],
       ['6', '', ''],
       ['7', '', ''],
       ['8', '', ''],
       ['9', '', '']], 
      dtype='<U32')
In [53]: np.savetxt('test.txt',arr, fmt='%5s',delimiter=',')
In [54]: cat test.txt
    0,  0.0,    0
    1,  0.1,   10
    2,  0.2,   20
    3,  0.3,     
    4,  0.4,     
    5,     ,     
    6,     ,     
    7,     ,     
    8,     ,     
    9,     ,

edited Jan 19, 2017 at 3:21

answered Jan 19, 2017 at 2:13

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Isaac Over a year ago

The reason I did v.T is to the 3 arrays as columns, Thank you for answers.

hpaulj Over a year ago

i've added examples using `itertools.zip_longest' (PY3) to form 'padded' columns.

Isaac Over a year ago

Thank you hpaulj. It really helps.

Collectives™ on Stack Overflow

save a multidimensional array with different sub-array length python

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related