Efficient binary encoding of struct

Question

I have a program that sends udp packets. a packet datagram is represented by the following struct (notice that the size of the packet isn't fixed):

type packet struct {
    dataLength uint16
    id [8]byte
    pairity, shards, flags byte
    blob []byte // length defined by dataLength 
}

I used encoding/binary package to do serialization/deserialization but it wasn't efficient enough (used pprof). it wasted alot of CPU time and I couldn't utilize the whole network speed because of that.

for example, consider the following code:

packet := packet{
    dataLength: 4,
    id: [8]byte{1,2,3,4,5,6,7,8},
    pairity: 10,
    shards: 50,
    flags: 200,
    blob: []byte{1,2,3,4},
}
bufToSendOverNetwork := packet.ToBytes()

What is the most efficient way to do this operation (and also the .FromBytes operation)

Can you show the binary encoding you're currently using, and where your bottleneck is? Since blob is a slice, your Go struct isn't something that be directly written to match the packet struct. — Mr_Pink
– Mr_Pink, Commented Mar 25, 2016 at 19:33

score 4 · Accepted Answer · 2016-03-25 19:39:21Z

4

encoding/binary.Write uses reflection, so it will be slower than anything that manually constructs the buffer. Below is an example of such a function:

import (
    "encoding/binary"
)

func (p *packet) ToBytes() []byte {
    buff := make([]byte, 2 + 8 + 3 + len(p.blob))

    binary.BigEndian.PutUint16(buff[:2], p.dataLength)
    copy(buff[2:10], p.id[:])
    buff[10] = p.pairity
    buff[11] = p.shards
    buff[12] = p.flags
    copy(buff[13:], p.blob)

    return buff
}

Collectives™ on Stack Overflow

Efficient binary encoding of struct

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related