Reading binary data written using Python struct in C

Question

I'm writing a binary file in Python to be read in C. The (MWE) code to write the file is:

import struct

with open('test.bin', 'wb') as outfile:  
    outfile.write(struct.pack('didi', 1.2, 1, 1.3, 2))

When I read the file in C, I get garbled data:

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>

int main(int argc, char *argv[]) {
    double testdouble, testdoubletwo;
    int testint, testinttwo;

    FILE *f = fopen("test.bin", "rb");
    assert(f);

    assert(fread(&testdouble, sizeof(testdouble), 1, f));
    assert(fread(&testint, sizeof(testint), 1, f));
    assert(fread(&testdoubletwo, sizeof(testdoubletwo), 1, f));
    assert(fread(&testinttwo, sizeof(testinttwo), 1, f));

    fprintf(stderr, "testdouble: %f, testint: %d, testdouble: %f, testinttwo: %d", testdouble, testint, testdoubletwo, testinttwo);

    return 0;
}

Output:

testdouble: 1.200000, testint: 1, testdouble: -92559641157289301412905710012271939667257667601819249288413184.000000, testinttwo: 1073007820

If I leave out the integers, it works for this small example, but not for my actual problem where I'm reading a few dozen doubles. Some of them (not the first, not the last) end up garbled.

System: Ubuntu 12.04, 64bit
Python: 2.7.3

Can you include a hex dump of the test.bin file that is produced by your Python code? — bdesham
– bdesham, Commented Dec 2, 2013 at 17:24
May have to do with alignment. Since you did not specifiy an alignment character @ is assumed. This shows native alignment. This would work if you had a c structure you were reading the entire thing. Try = struct.pack('=d=i=d=i'.....) — Robert Jacobs
– Robert Jacobs, Commented Dec 2, 2013 at 18:19
Are you writing and reading on the same machine, or at least the same endianness? Or is data portability a requirement? — Robᵩ
– Robᵩ, Commented Dec 2, 2013 at 21:39
@RobertJacobs: almost the solution (see below), but the '=' does not need to be and cannot be repeated. Problem solved, thanks everyone! — Johannes Bauer
– Johannes Bauer, Commented Dec 3, 2013 at 11:07

PasteBT · Accepted Answer · 2013-12-02 19:03:41Z

1

In your C code, you read every item out one by one, which means you did not apply any alignment. Try this:

outfile.write(struct.pack('=didi', 1.2, 1, 1.3, 2))

hexdump test.bin

0000000 3333 3333 3333 3ff3 0001 0000 cccd cccc
0000010 cccc 3ff4 0002 0000

C code output:

testdouble: 1.200000, testint: 1, testdouble: 1.300000, testinttwo: 2

If you not change python code, still use 'didi', then change c code like this:

struct D {
    double td;
    int ti;
    double td2;
   int ti2;
};


struct D d;
fread(&d, sizeof(struct D), 1, f);
fprintf(stderr, "testdouble: %f, testint: %d, testdouble: %f, testinttwo: %d", d.td, d.ti, d.td2, d.ti2);

This test on Fedora 17, using python 2.7.3, gcc 4.7.2, I prefer define the structure.

edited Dec 2, 2013 at 19:03

answered Dec 2, 2013 at 18:47

PasteBT

2,20817 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Heng Li · Accepted Answer · 2020-07-09 12:25:37Z

I should also say that it is possible to change the C program and not the python program to work. This is almost perfect except that on my system python writes a 28 byte file, and my C program expects 32 bytes. It works without the assert. Not ideal. Writing a junk int after testinttwo would fix the problem.

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>

int main(int argc, char *argv[]) {
    typedef struct { double testdouble;
                     int testint;
                     double testdoubletwo;
                     int testinttwo;
                } teststruct;
    teststruct abc;
    FILE *f = fopen("test.bin", "rb");
    assert(f);
/*  assert(fread(&abc, sizeof(teststruct), 1, f));*/
    printf("sizeof(teststruct) %d\n",sizeof(teststruct));
    fread(&abc, sizeof(teststruct), 1, f);

    fprintf(stderr, "testdouble: %f, testint: %d, testdouble: %f, testinttwo: %d", abc.testdouble, abc.testint, abc.testdoubletwo, abc.testinttwo);

    fclose(f);
    return 0;
}

Adjusted python program to fix the assert issue.

import struct

with open('test.bin', 'wb') as outfile:
    outfile.write(struct.pack('didii', 1.2, 1, 1.3, 2,0))

C Output:

sizeof(teststruct) 32
testdouble: 1.200000, testint: 1, testdouble: 1.300000, testinttwo: 2

Ishraga Mustafa Awad Allam · Accepted Answer · 2022-03-08 08:26:03Z

0

//  BY: Ishraga Mustafa Awad Allam. ON: 22-2-2020

#include <stdio.h>
#include <stdlib.h>

struct rec {
    int i;
    float r;
    char str[10];
    }rc;

int main(){
   FILE *f1;
   float f;
   int j = 0;


f1 = fopen("c:\\code\\python\\dfile.dat", "r");// or exit("Unable to open file!");

  while(!feof(f1)) {
     j++;
     fread(&rc, sizeof(rec), 1, f1);
     printf("%4d %4d %12.6f %s \n", j, rc.i, rc.r, rc.str);

     }

printf("\nGOOD TERMINATION \n");

fclose(f1);


getchar();
return 0;
}

answered Mar 8, 2022 at 8:26

Ishraga Mustafa Awad Allam

111 bronze badge

2 Comments

Community Over a year ago

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

Muhammad Mohsin Khan Over a year ago

Please explain what your code does as well.

Collectives™ on Stack Overflow

Reading binary data written using Python struct in C

3 Answers 3

Comments

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related