2

I have to dump the contents of a numpy ndarray to a binary file which is going to be read by a third party program. However, what I want to do is write the contents of a permuted axes. As an example, I have something like:

import numpy as np
x = np.random.rand(3, 3, 3)

a = np.transpose(x, (1, 0, 2))
a.tofile("a.bin")
x.tofile("x.bin")

In both the cases, the outputted files are the same. Is there a transpose like operation which willa ctually move the contents of the array around rather than just swap the stride and dimensions? This way the raw content would be serialized in the order that I need.

Improve this question

2 Answers 2

Reset to default
2

I cannot reproduce your findings: I get different arrays:

In [11]: np.fromfile('a.bin').reshape((3,3,3))
Out[11]: 
array([[[0.95499073, 0.53044188, 0.31122484],
        [0.44293225, 0.23932913, 0.13954034],
        [0.08992127, 0.59397388, 0.72471928]],

       [[0.43503453, 0.15910105, 0.10589887],
        [0.39610877, 0.68784233, 0.87956587],
        [0.89785046, 0.64688383, 0.40787343]],

       [[0.91490793, 0.31428658, 0.85234109],
        [0.36403572, 0.99601086, 0.46086401],
        [0.43524914, 0.85182394, 0.01254642]]])

In [12]: np.fromfile('x.bin').reshape((3,3,3))
Out[12]: 
array([[[0.95499073, 0.53044188, 0.31122484],
        [0.43503453, 0.15910105, 0.10589887],
        [0.91490793, 0.31428658, 0.85234109]],

       [[0.44293225, 0.23932913, 0.13954034],
        [0.39610877, 0.68784233, 0.87956587],
        [0.36403572, 0.99601086, 0.46086401]],

       [[0.08992127, 0.59397388, 0.72471928],
        [0.89785046, 0.64688383, 0.40787343],
        [0.43524914, 0.85182394, 0.01254642]]])

Without reshape:

In [22]: np.fromfile('a.bin')
Out[22]: 
array([0.95499073, 0.53044188, 0.31122484, 0.44293225, 0.23932913,
       0.13954034, 0.08992127, 0.59397388, 0.72471928, 0.43503453,
       0.15910105, 0.10589887, 0.39610877, 0.68784233, 0.87956587,
       0.89785046, 0.64688383, 0.40787343, 0.91490793, 0.31428658,
       0.85234109, 0.36403572, 0.99601086, 0.46086401, 0.43524914,
       0.85182394, 0.01254642])

In [23]: np.fromfile('x.bin')
Out[23]: 
array([0.95499073, 0.53044188, 0.31122484, 0.43503453, 0.15910105,
       0.10589887, 0.91490793, 0.31428658, 0.85234109, 0.44293225,
       0.23932913, 0.13954034, 0.39610877, 0.68784233, 0.87956587,
       0.36403572, 0.99601086, 0.46086401, 0.08992127, 0.59397388,
       0.72471928, 0.89785046, 0.64688383, 0.40787343, 0.43524914,
       0.85182394, 0.01254642])

My version of numpy and Python are:

In [21]: import sys
    ...: print(sys.version)
    ...: print("\nNumpy version: " + np.__version__)
    ...: 
2.7.15 |Anaconda, Inc.| (default, May  1 2018, 18:37:05) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]

Numpy version: 1.14.5

I also tried a different environment with the same result:

In [1]: import sys
   ...: import numpy as np
   ...: print(sys.version)
   ...: print(np.__version__)
   ...: 
3.6.5 |Anaconda, Inc.| (default, Apr 26 2018, 08:42:37) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
1.13.3

In [2]: x = np.random.rand(3, 3, 3)
   ...: a = np.transpose(x, (1, 0, 2))
   ...: a.tofile("a.bin")
   ...: x.tofile("x.bin")
   ...: 

In [3]: np.fromfile('a.bin').reshape((3,3,3))
Out[3]: 
array([[[ 0.7628757 ,  0.5117887 ,  0.85286206],
        [ 0.27096479,  0.5056376 ,  0.14519906],
        [ 0.9517039 ,  0.92225717,  0.85885034]],

       [[ 0.57380259,  0.74694459,  0.19207375],
        [ 0.50738877,  0.33581015,  0.57100872],
        [ 0.54989565,  0.35004858,  0.9527302 ]],

       [[ 0.94359803,  0.6223541 ,  0.57774136],
        [ 0.92983442,  0.98074324,  0.62467311],
        [ 0.49712549,  0.73399765,  0.56790972]]])

In [4]: np.fromfile('x.bin').reshape((3,3,3))
Out[4]: 
array([[[ 0.7628757 ,  0.5117887 ,  0.85286206],
        [ 0.57380259,  0.74694459,  0.19207375],
        [ 0.94359803,  0.6223541 ,  0.57774136]],

       [[ 0.27096479,  0.5056376 ,  0.14519906],
        [ 0.50738877,  0.33581015,  0.57100872],
        [ 0.92983442,  0.98074324,  0.62467311]],

       [[ 0.9517039 ,  0.92225717,  0.85885034],
        [ 0.54989565,  0.35004858,  0.9527302 ],
        [ 0.49712549,  0.73399765,  0.56790972]]])
Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

which version of numpy are you using? Did you use transpose as well?
@Luca I just run your code exactly as you posted it.
I am on 1.14.3 as well. Can you just try the output the np.fromfile("a.bin") and np.fromfile("x.bin") as well without the reshape. I am not sure why we are getting different results
I ran your code and I am getting the same array back. Only thing I can tbink of is that for some reason your version is not returning a view into the array...
Yes, I created another environment with the same version of numpy and it was working for some reason. I have no clue how this can happen. I reran the code 10s of times just to be sure. I have a deadline for Sunday but after that I will try to get to the bottom of this. It is most inexplicable...
|
1

I think you just need to use numpy's swapaxes.

import numpy as np
x = np.random.rand(3, 3)
a = np.swapaxes(x,0,1)
a.tofile("a.bin")
x.tofile("x.bin")

Hope this helps!

Improve this answer

5 Comments

Hi, yes, my example was a bit contrived. But imagine you have something like... x = np.random.rand(3, 3, 3) and then you just want to swap the first two axes. I edited the question to reflect that.
Hi, I have updated the code, you need to use np.swapaxes.
This also, unfortunately, returns a view. I am using the latest version of numpy. Also from the documentation: For NumPy >= 1.10.0, if a is an ndarray, then a view of a is returned; otherwise a new array is created
x.T is not what I need. As I only need to swap a couple of the axes not the whole array...
swapaxis and transpose can do the same thing

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.