Write Bin File Using Python C-module And Read The File In C

Question

I'm using ctypes to convert some info in a binary file that needs to be read by a C++ program. This file will contain lines with strings and others with double/vectors of double.

The problems occur when I try to read a vector of double in C++: the address of the vector goes in the end of the line and the address has 11 char and in C++ has 8 char. When I try to read directly the vector into C++ an error occurs due to this difference.

A possible solution would be to read element by element of the vector, but it will cost performance.

There is any possibility to rip the address during the conversion and reading, or ignore it, and the whole vector at once?

Here is some code we did to test:

C++ DLL Module:

#include<iostream>
#include<fstream>
#include<cstring>
#include <typeinfo>

using namespace std;

#define DLLEXPORT extern "C" __declspec(dllexport)

struct new_element {
    int id;
    unsigned int n_measures;
    double* value;
};

DLLEXPORT int writer_bin_file_c(const char* file_path, new_element structed_data)
{
    try {
        ofstream output_file(file_path);
        output_file.write((char*)&structed_data, sizeof(structed_data));  
        output_file << &structed_data;
        output_file.close();
    } catch (...) {
        return -1;
    }

    return 0;
}

DLLEXPORT new_element reader_bin_file_c(const char* file_path, new_element structed_data)
{
    try {
        ifstream input_file(file_path, ios::binary);
        input_file.read((char*)&structed_data, sizeof(structed_data));       
        input_file.close();
    } catch (...) {
        cout << "Error ao ler o arquivo" << endl;
    }

    return structed_data;
}

Python writing file:

from ctypes import *
import sys
import numpy as np

lib = CDLL("version4/template_file.so")

class new_element(Structure):
    _fields_ = [("id", c_int),
                ("n_measures", c_uint),
                ("value", POINTER(c_double))]

template_file = lib
new_element_pointer = POINTER(new_element)

writer_bin_file = template_file.writer_bin_file_c
writer_bin_file.argtypes = [c_char_p, new_element]
writer_bin_file.restype = c_void_p 

reader_bin_file_c = template_file.reader_bin_file_c
reader_bin_file_c.restype = new_element

tam = 10
medida = np.arange(tam, dtype=c_double)
medida = medida.ctypes.data_as(POINTER(c_double))

element = new_element(4, tam)
element.value = medida

file_out = b'version4/element.txt'

answer = writer_bin_file(file_out, element)

C++ reading file:

#include<iostream>
#include<fstream>
#include<cstring>
#include <typeinfo>

using namespace std;

struct new_element {
    int id;
    unsigned int n_measures;
    double* value;
};

new_element reader_bin_file(const char* file_path, new_element structed_data)
{
    try {
        ifstream input_file(file_path);
        input_file.read((char*)&structed_data, sizeof(structed_data));    
        input_file.close();
    } catch (...) {
        cout << "Error ao ler o arquivo" << endl;
    }

    return structed_data;
}

int main(int argc, char const *argv[])
{
    new_element read_template;
    read_template = reader_bin_file(file_out, read_template);

    cout << "ID: " << read_template.id << endl;
    cout << "n_measures: " << read_template.n_measures << endl;
    cout << "Value: ";
    for (int i = 0;  i < read_template.n_measures; i++) 
    {
      cout << "( " << i << " ): " << read_template.value[i] << " | ";
    }
    cout << endl;

    return 0;
}

Start by opening your files in binary mode; will make a difference if you're using Windows, good practice on other OSes. — Shawn
– Shawn, Commented Oct 16, 2020 at 18:59

Viktor Latypov · Accepted Answer · 2020-10-16 19:35:59Z

1

You have one deep misunderstanding here.

The pair of fields

unsigned int n_measures;
double* value;

is an array (deducing from their names), so in a writer you have to save n_measures double value, not one single pointer. Consequently, in the reader you should read n_measures values, not just a pointer. A pointer is just an index in a memory space, not some "all-powerful" C/C++ language feature, which saves everything you need.

So, in your C++ writing code do

DLLEXPORT int writer_bin_file_c(const char* file_path, new_element structed_data)
{
  try {
    ofstream output_file(file_path);
    output_file.write((char*)&structed_data.id, sizeof(int));
    output_file.write((char*)&structed_data.n_measures, sizeof(int));
    // write out all the elements one by one, not just the pointer
    for (int i = 0 ; i < structed_data.n_measures ; i++)
       output_file.write((char *)&structed_data[i], sizeof(double));

    output_file.close();
} catch (...) {
    return -1;
}

return 0;

}

Hope you get the idea, at least on the C++ part. The reader code is similar - read id, n_measures and then allocate the values array and read values one by one.

answered Oct 16, 2020 at 19:35

Viktor Latypov

14.5k3 gold badges44 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Luiz Carlos Macedo Over a year ago

I'll try it and give you a feedback soon, thanks for your answer.

Luiz Carlos Macedo Over a year ago

We did a approach like that, with some modifications to the real code and we get it. Thanks man!

Collectives™ on Stack Overflow

Write Bin File Using Python C-module And Read The File In C

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related