Cannot read integer values saved into binary file with Java

Question

I implemented a procedure for saving 32bit signed integer values, retrieved from PostgreSQL, into a binary file.

I used ByteArrayOutputStream and DataOutputStream

//..


ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream out = new DataOutputStream(baos);

//this cycle over my retrieved recordset
while (tdb.next()) {

   out.writeInt((tdb.getInteger(1)));  //getInteger method extract integer value from recordset

}

//convert into byte array
byte[] bytes = baos.toByteArray();

//create file for output
File fileout = new File("./myfilebin.bin");
//write data
Files.write(fileout.toPath(), bytes);

//...

My input data sample contains these values:

0, 0, 42812, 42822, 41483, 0, 0, ...

When I try to read my binary file, I will get these:

0, 0, 1017577472, 1185349632, 195166208, 0, 0, ...

For reading file I wrote a simple python script:

import struct

with open("./myfilebin.bin", "rb") as f:

     for chunk in iter(lambda: f.read(4), ''):
        integer_value = struct.unpack('i', chunk)[0]
        print(integer_value)

Any idea?

How do you know those are the actual input numbers? Making 5398449 out of 48 does not appear possible here. The python code reads LE, the javacode writes BE, so that's a problem, but you'd get 805306368, not 5398449. — rzwitserloot
– rzwitserloot, Commented May 18, 2022 at 23:33
Why the ByteArrayOutputStream? Just wrap the DataOutputStream around the FileOutputStrream and write directly to the file. If you need it quicker, put a BufferedOutputStream in between them. — user207421
– user207421, Commented May 18, 2022 at 23:43
@rzwitserloot I make it 11624274. OP needs to provide a hex dump of the file. — user207421
– user207421, Commented May 18, 2022 at 23:47
yes, the problem was the different behaviour between javacode and python code for endian management — N3tMaster
– N3tMaster, Commented May 19, 2022 at 8:51
So what is the correct sample? You can't expect a solution if you don't provide all the relevant data. — user207421
– user207421, Commented May 20, 2022 at 0:15

Michail Alexakis · Accepted Answer · 2022-05-18 22:43:06Z

2

Probably your numbers are Big-Endian (BE) and you try to decode as Little-Endian (LE).

Try: struct.unpack('>i', chunk) to force reading as BE.

See also the docs for struct module on alignment and endianess.

FYI, DataOutputStream always writes in BE as stated in the official javadoc.

edited May 18, 2022 at 22:43

answered May 18, 2022 at 22:33

Michail Alexakis

1,60516 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

N3tMaster Over a year ago

if I change the decode option, output changes but still wrong numbers

Michail Alexakis Over a year ago

@N3tMaster, can you share a sample of the binary file. For example, for first 32 bytes: head -c 32 file.bin | xxd.

Michail Alexakis Over a year ago

@user207421, both links are correct; the 1st link states that bytes are written "high byte first" or in other words big-endian.

N3tMaster · Accepted Answer · 2022-05-19 08:54:41Z

0

Ok, the problem was the different behaviour in bit reading order between python, that uses LE, and java.

I need to create a binary file that follows ISOBUS standards and I have to write it using LE.

So I change my code in this way:

FileOutputStream fout = new FileOutputStream("myfileout.bin");

            DataOutputStream out = new DataOutputStream(fout);


            while (tdb.next()) {
                //convert byte endian behaviour for ISOBUS software compatibility
                ByteBuffer bb = ByteBuffer.allocate(4);
                bb.order(ByteOrder.LITTLE_ENDIAN);
                bb.putInt(tdb.getInteger(1));
                //invert significant byte
                bb.flip();

                out.write(bb.array());
            }

            fout.close();
            out.close();

answered May 19, 2022 at 8:54

N3tMaster

4934 silver badges15 bronze badges

2 Comments

Michail Alexakis Over a year ago

you code is correct, but your comment on flip is wrong: flip does not change endianess, just sets the position to 0 so that you can read what you have previously written on the buffer

N3tMaster Over a year ago

you are right, but te final result is a switched bytes (BE with LE) when data is written out

Collectives™ on Stack Overflow

Cannot read integer values saved into binary file with Java

2 Answers 2

3 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related