python save array to csv with different length

Question

this is my code

import numpy
a = numpy.asarray([ [1,2,3,4],[5,6],[7,8,9,10] ])
numpy.savetxt("a.csv", a, fmt="%d", delimiter=",")

I run it, but it report error

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 1422, in savetxt
    v = format % tuple(row) + newline
TypeError: %d format: a number is required, not list

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/Desktop/csvTest.py", line 3, in <module>
    numpy.savetxt("foo.csv", a,fmt="%d", delimiter=",")
  File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 1426, in savetxt
    % (str(X.dtype), format))
TypeError: Mismatch between array dtype ('object') and format specifier ('%d')

after i try this code

import numpy
a = numpy.asarray([ [1,2,3],[4,5,6],[7,8,9] ])
numpy.savetxt("a.csv", a, fmt="%d", delimiter=",")

it can work, so how do i to solve this problem? can i save different array length with csv?

look at a. Is it a 2d array, with consistent rows and columns? Or a 1d array containing lists? I suggest forgetting numpy, and writing the list of lists directly to the file. Use regular python code to format the lists as you want. By the way, how do you expect to read this file? — hpaulj
– hpaulj, Commented Jul 1, 2021 at 4:25
Actually this issue i am not very care, it just from my curiosity. — 李家峰
– 李家峰, Commented Jul 1, 2021 at 6:03

AJITH · Accepted Answer · 2021-07-01 04:15:41Z

1

Saving the array as string can resolve the issue.

a = numpy.asarray([ [1,2,3,4],[5,6],[7,8,9,10] ])
numpy.savetxt("a.csv", a, fmt="%s", delimiter=",")

answered Jul 1, 2021 at 4:15

AJITH

1,1757 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

hpaulj · Accepted Answer · 2021-07-01 04:54:12Z

0

In [98]: alist=[[1,2,3,4],[5,6],[7,8,9,10] ]
In [99]: alist
Out[99]: [[1, 2, 3, 4], [5, 6], [7, 8, 9, 10]]

formatting a display string:

In [100]: astr = []
     ...: for row in alist:
     ...:     col = ', '.join(['%d'%i for i in row])
     ...:     astr.append(col)
     ...: astr = '\n'.join(astr)
In [101]: astr
Out[101]: '1, 2, 3, 4\n5, 6\n7, 8, 9, 10'
In [102]: print(astr)
1, 2, 3, 4
5, 6
7, 8, 9, 10

and with a file write:

In [104]: with open('test.txt','w') as f:
     ...:     for row in alist:
     ...:         print(', '.join(['%d'%i for i in row]), file=f)
     ...: 
In [105]: cat test.txt
1, 2, 3, 4
5, 6
7, 8, 9, 10

If we try to write it as an array:

In [106]: arr = np.array(alist)
<ipython-input-106-3fd8e9bd05a9>:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  arr = np.array(alist)
In [107]: arr
Out[107]: 
array([list([1, 2, 3, 4]), list([5, 6]), list([7, 8, 9, 10])],
      dtype=object)

This is a 1d array containing lists. If we write with '%s' format the lists will be formatted as standard lists:

In [108]: np.savetxt('test.csv',arr, fmt='%s')
In [109]: cat test.csv
[1, 2, 3, 4]
[5, 6]
[7, 8, 9, 10]

Neither is good csv reading material. The [] interfere with parsing a line. The unequal lines require special handling, either rejection as being the wrong length, or filled with extra values.

answered Jul 1, 2021 at 4:54

hpaulj

233k14 gold badges260 silver badges392 bronze badges

2 Comments

李家峰 Over a year ago

thx answer me how to solve, it a little complex... but it only save string? can change it to int?

hpaulj Over a year ago

The '%s' turns anything into a string. But all the other Python formats are available, And if you are writing directly (not with savetxt) you can use the new format tool.

Collectives™ on Stack Overflow

python save array to csv with different length

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related