Java: Convert byte stream to objects very efficiently

Question

I have a byte[] which contains serialised C++ data types (so it has a weird format that the standard Java serialisation classes can't understand). I've created my own deserialiser which matches flags to types and converts it into the appropriate object, like so (input could be something like this int64 with value 4 [3, 0, 0, 0, 0, 0, 0, 0, 4], 3 is the flag for int64)

private final ByteBuffer byteWrapper;

// member ByteBuffer which is allocated in the ctor

ctor { byteWrapper = ByteBuffer.allocateDirect(8); }

// somewhere in the parse function, this case matches an int64
case 3:
{
        return byteWrapper.wrap(ios.getBytes(8)).getLong();
}

Where ios contains the full serialisation data for easy reading. Now this works fine, but high throughput is very important for this piece of the code and I'm not a regular Java programmer, so I'm just wondering if anyone has any suggestions for speeding this process up, either by reworking what I've got or using different classes/functions entirely or anything? Memory is not an issue, I'm just interested in speed

Mario Dekena · Accepted Answer · 2016-04-01 10:53:21Z

1

Dont copy your data into a new buffer object. Read it directly from the ios buffer.

byte type = ios.get();
switch (type) {
    case 3:
        return ios.getLong();
}

All getters in ByteBuffer will increase the position according to the length of the data type. get() increases the position by 1, getLong() by 8, getFloat() by 4... You can directly use ios.getLong(). Assuming that the values are written one after the other, the new position will point to the next type-byte. So you can directly call ios.get() to get the next type.

Edit:

If ios is an InputStream you can use DataInputStream to read. It also supports various datatypes.

DataInputStream input = new DataInputStream(ios);

byte type = input.readByte();
switch (type) {
case 3:
    return input.readLong();
}

In Java, there are essentially two packages, that provide the functionality you are looking for. java.io and java.nio. java.io contains Streams, while java.nio contains Buffers. java.nio is faster, because it uses very low level memory operations. If you are looking for speed, you should use that. That also means you should change the type of ios to ByteBuffer if possible and then use my first approach.

edited Apr 1, 2016 at 10:53

answered Apr 1, 2016 at 9:50

Mario Dekena

8436 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Madden Over a year ago

That sounds good but it looks like my ios (java.io.InputStream) doesn't have that method, is there a different stream I should be using?

Mario Dekena Over a year ago

@Madden edited once more theres something you should consider.

Collectives™ on Stack Overflow

Java: Convert byte stream to objects very efficiently

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related