Serializing struct to file, and deserializing it again with string

Question

I have a following struct :

struct ArchiveTableEntry
        {
            uint32_t id;
            char filename[MAX_FILENAME_LEN + 1];
            uint32_t filename_crc32;
            uint32_t real_data_size;
            uint32_t block_data_size;

            uint32_t data_crc32;
            char data_md5_checksum[16];

            uint64_t data_offset;
            uint8_t flag;
        };

And after serialization of it I have good output to file. Also loading from file work propertly too. I use memcpy to perform serialization/deserialization, and now my question is : How can I replace char filename[MAX_FILENAME_LEN + 1]; by std::string, and keep serialization/deserialization working with save unused bytes that char waste. Thanks!

struct fields might be underlie memory aligned padding ... Use an appropriate serialization mechanism to get size efficient outputs (e.g. google protobuf). — πάντα ῥεῖ
– πάντα ῥεῖ, Commented Jan 24, 2014 at 23:51

Brandon · Accepted Answer · 2014-01-25 00:07:13Z

3

Try to avoid using memcpy-ing to a pointer for serialization.. Instead, make use of streams.. You can even make a stream class that serializes any type of data. Next you'll just have to add << and >> operators to any class that needs to be serialized.

#include <iostream>
#include <fstream>
#include <string>
#include <cstring>

struct ArchiveTableEntry
{
    uint32_t id;
    std::string filename;
    uint32_t filename_crc32;
    uint32_t real_data_size;
    uint32_t block_data_size;

    uint32_t data_crc32;
    char data_md5_checksum[16];

    uint64_t data_offset;
    uint8_t flag;
};

std::ostream& operator << (std::ostream& os, const ArchiveTableEntry entry)
{
    return os << entry.id << entry.filename.size() << entry.filename
              << entry.filename_crc32 << entry.real_data_size << entry.block_data_size
              << entry.data_crc32 << entry.data_md5_checksum << entry.data_offset << entry.flag;
}

std::istream& operator >> (std::istream& os, ArchiveTableEntry& entry)
{
    std::size_t filesize = 0;
    os >> entry.id;
    os >> filesize;
    entry.filename.resize(filesize);
    os.read(&entry.filename[0], filesize);
    os >> entry.filename_crc32 >> entry.real_data_size >> entry.block_data_size >> entry.data_crc32;
    os >> entry.data_md5_checksum >> entry.data_offset >> entry.flag;
    return os;
}



int main()
{
    const char* md5_checksum = "some checksum";

    ArchiveTableEntry data;
    data.id = 1;
    data.filename = "file.txt";
    data.filename_crc32 = 10434235;
    data.real_data_size = 1024;
    data.block_data_size = 256;
    data.data_crc32 = 324225252;
    data.data_md5_checksum = 1;

    std::memset(data.data_md5_checksum, 0, sizeof(data.data_md5_checksum) / sizeof(char));
    strcpy(data.data_md5_checksum, md5_checksum);

    data.data_offset = 512;
    data.flag = 1;

    std::fstream out("C:/Users/School/Desktop/Test.bin", std::ios::out);
    if (out.is_open())
    {
        out << data;
        out.close();
    }

    std::fstream in("C:/Users/School/Desktop/Test.bin", std::ios::in);
    if (in.is_open())
    {
        in >> data;
        in.close();
    }

    std::cout<<data.id<<" "<<data.filename<<" "<<data.filename_crc32<<" "<<data.real_data_size<<" "<<data.block_data_size<<" ";
    std::cout<<data.data_crc32<<" "<<data.data_md5_checksum<<" ";
}

edited Jan 25, 2014 at 0:07

answered Jan 24, 2014 at 23:50

Brandon

23.6k13 gold badges110 silver badges200 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Tara Over a year ago

"Try to avoid using memcpy-ing to a pointer for serialization." Yes, if you hate performance.

Brandon Over a year ago

Lol.. OP was creating a buffer, copying everything to the buffer and writing that to the file. Secondly, he cannot memcpy an std::string, which was his issue in the first place. He wasn't writing the length of the string to the file and then the contents. With streams, he doesn't have to copy to a temporary buffer and then write to the file. It also makes his life easier when using other standard library classes.

Tara Over a year ago

You're right about the std::string part. But usually you do serialization/deserialization using a structure you can read/write in one go. This is usually faster than multiple small read/write operations.

Collectives™ on Stack Overflow

Serializing struct to file, and deserializing it again with string

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related