Serialize array data from C code, deserialize in Python

I've done some serialization for arm devices and network and I'would be happy to share my experience since you prefer binary serialization. I am using unions to serialize.Lets assume that you have a struct that holds some element, data and pointers and this holds data for a family member :

struct fam_member
{
    char name [ MAX_NAME_LEN + 1 ];
    int height;
    age_bracket_t age_bracket;
    fam_member_t* mother;
    fam_member_t* father;
}fam_member_t;

Age bracket is an enum:

typedef enum age_bracket
{
    under_18 = 0 , from_18_to_25 = 1 , from_26_to_40 = 2  , over_40 = 3
}age_bracket_t;

The main problem and the most common mistake is struct padding and not taking this to serious.Here is a good start if someone is not familiar with the issue. My simple solution is stream data down byte to byte (or bit to bit), do what you need to do with the serialized data (i.e. send them over a socket) and deserialize in the end. I define a Union like this:

typedef union serialized_struct
{
    fam_member_t family_member;
    unsigned char data[ (MAX_NAME_LEN + 1 ) + (sizeof(int)*3) ];

}serialized_struct_t;

(A few think about union here) The purpose of union is to save memory by using the same memory region for storing different objects at different times.In this example this will help us and actually serialize the family object struct for free.

Here is a function that serializes an array of family members (if you can do an area, single will be a piece of cake.That's why I choose an array here).

int serialize_array(fam_member_t* people , char* message , int elements)
{
    if((people == NULL ) || (message == NULL) || (elements < 1))
    {
        return -1;
    }
    int size = sizeof(fam_member_t);
    int i;
    for(i=0 ; i < elements ; i++)
    {
        serialized_struct_t x;
        memcpy((x.family_member.name) , people[i].name , MAX_NAME_LEN);
        x.family_member.age_bracket = people[i].age_bracket;
        x.family_member.height = people[i].age_bracket
        x.family_member.mother = people[i].mother;
        x.family_member.father = people[i].father;
        memcpy ( (message + (size * i)) , x.data , size );
    }
    return 0;
}

Here we initiate every data of every member inside the struct which lies in the union.Message holds serialized data.This is the deserialized function which will do the reverse

int desirialize_array(fam_member_t* people , char* message , int elements)
{
    if((people == NULL ) || (message == NULL) || (elements < 1))
    {
        return -1;
    }
    int size = sizeof(fam_member_t);
    serialized_struct_t y;
    int i;

    for (i =0 ; i < elements ; i ++ )
    {
        memcpy ( y.data , (message + (size * i)) , size );
        memcpy ( people[i].name , y.family_member.name , MAX_NAME_LEN);
        people[i].age_bracket = y.family_member.age_bracket;
        people[i].height = y.family_member.height;
        people[i].mother = y.family_member.mother;
        people[i].father = y.family_member.father;
    }
    return 0;
}

This is serialize and deserialize in c example.For your case where you need to deserialize this in python I think it will be easy if you figured out which will be the mean of serialization.JSON that @Alexander Tolkachev said for example could be a solution. I hope this simplified example helps you.

This repository contains that you need!

https://github.com/souzomain/Packer

from Packer import Packer, Parser
packer = Packer()
packer.add_str("hello world")
print(f"packet size: {packer.get_size()} | packet: {packer.get_buffer()}")