error using memmap to parse header in python numpy

Question

Here is the code that i am trying to enforce with real example

#Create a dtype to represent the header.
header_dtype = dtype([('rows', int32), ('cols', int32)])
# Create a memory mapped array using this dtype. Note the shape is empty.
header = memmap(file_name, mode='r', dtype=header_dtype, shape=())
# Read the row and column sizes from using this structured array.
rows = header['rows']
cols = header['cols']
# Create a memory map to the data segment, using rows, cols for shape
# information and the header size to determine the correct offset.
data = memmap(file_name, mode='r+', dtype=float64,shape=(rows, cols),  
offset=header_dtype.itemsize)

The file format that i want to read using memmap is

enter image description here

I am wondering what is going wrong when i try to read the following file named matrix.dat

4 4
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16

I am doing the following in python (Enthought's canopy editor)

header_dtype = dtype([('rows',int),('cols',int)])
header=memmap('matrix.dat',mode='r',dtype=header_dtype,shape=())
rows = header['rows']
cols=header['cols']
print header
(221519924, 840970506)

print rows cols
221519924, 840970506

Why i am not getting 4, 4? How do i change the file matrix.dat shown above to read the data correctly?

The file is understood as binary, so the first 4 bytes will be 0x3420340d, which translates to 221519924. You probably have to write the values in binary mode in the file. — Yolanda Ruiz
– Yolanda Ruiz, Commented Feb 22, 2014 at 21:52
@YolandaRuiz You should post that as an answer. With the addition that numpy.loadtxt if better suited for text files. — Roland Smith
– Roland Smith, Commented Feb 23, 2014 at 0:20

Roland Smith · Accepted Answer · 2014-02-23 00:25:58Z

1

The file is understood as binary, so the first 4 bytes will be 0x3420340d, which translates to 221519924. You probably have to write the values in binary mode in the file.

For text files, use numpy.loadtxt instead;

In [1]: import numpy as np

In [2]: cat matrix.dat
4 4
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16

In [3]: np.loadtxt('matrix.dat')
Out[3]: 
array([[  4.,   4.],
       [  1.,   2.],
       [  5.,   6.],
       [  9.,  10.],
       [ 13.,  14.]])

edited Feb 23, 2014 at 0:25

Roland Smith

43.8k3 gold badges69 silver badges98 bronze badges

answered Feb 23, 2014 at 0:21

Yolanda Ruiz

7164 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user2979872 Over a year ago

Yolanda! thanks for your answer. I am looking for solution using memmap. Let me put the complete code so that you get to know every thing.

Collectives™ on Stack Overflow

error using memmap to parse header in python numpy

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related