Writing binary in c++ and read in python

Question

I would like to store a series of numbers to a binary file using c++, to open later using python. Currently I have the following script in c++ named *writer.cpp:

#include <fstream>

int main()
{
  std::ofstream outFile;
  outFile.open("binaryFile.bin", std::ios::binary);
  int number = 0;
  for (int i=0; i<10; i++){
    number += i;
    outFile.write(reinterpret_cast<const char*>(&number), sizeof(int));
  }
  outFile.close();

  return 0;
}

Which when compiling as

g++ -o writerTest.x writer.cpp

and run as

./writerTest.x

produce a binary file named "binaryFile.bin".

I then try to read this using python with the following script named reader.py:

import numpy as np
from sys import argv

path = str(argv[1])
x = np.fromfile(path)
print x, type(x)

Running this as python reader.py binaryFile.bin produce the following result

[  2.12199579e-314   1.27319747e-313   3.18299369e-313   5.94158822e-313
9.54898106e-313] <type 'numpy.ndarray'>

Which obviously is not what I was hoping for. What am I doing wrong, and how should this be done properly?

You cannot just cast and write into a file, you must serialize your data to a known format (an existing one, or one you create yourself). — spectras
– spectras, Commented May 28, 2016 at 20:14
You are writing integers and reading floats, that cannot work. (Binary) — Baum mit Augen
– Baum mit Augen ♦, Commented May 28, 2016 at 20:15
@spectras: In practice, assuming an optimizer that is not overly aggressive and a "reasonable" platform, OP's code is likely to work in most cases. In theory, you do have to bit-mask and reconstruct the thing into a series of characters, but meh. — Kevin
– Kevin, Commented May 28, 2016 at 20:17
@Kevin: …and that the python interpreter was compiled with the same compiler as his code. In the end, that's a lot of assumptions for anything more than a small proof of concept. — spectras
– spectras, Commented May 28, 2016 at 20:33

aldanor · Accepted Answer · 2016-05-28 20:17:44Z

6

You have to specify the type of the values you're going to be reading, numpy has no way to guess that as there is no metadata stored in the file itself. So in your case you have to do something like:

x = np.fromfile(path, dtype=int)

If you're doing things like this, it's highly recommended to use fixed-size integers instead of just int, e.g. in C++ you can use int32_t from <cstdint> and in Python you can specify int32 as dtype.

edited May 28, 2016 at 20:17

answered May 28, 2016 at 20:16

aldanor

3,4912 gold badges29 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

spectras Over a year ago

This, and you should also specify the byte order. On both sides.

filiphl Over a year ago

Thanks! had to specify dtype=np.int32. Worked like a charm. [ 0 1 3 6 10 15 21 28 36 45] <type 'numpy.ndarray'>

slugonamission · Accepted Answer · 2016-05-28 20:15:00Z

3

fromfile assumes floating point numbers by default. If you want to change this behaviour, you need to pass in the type in the dtype named parameter.

As you're writing ints, this should work:

x = np.fromfile(path, dtype=int)

answered May 28, 2016 at 20:15

slugonamission

9,6741 gold badge36 silver badges41 bronze badges

Collectives™ on Stack Overflow

Writing binary in c++ and read in python

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related