byte[] array to struct with variable length array

Question

I'm receiving an array of bytes from a socket and the structure of the bytes is simply a large char array of fixed width strings. In some cases, the last field is dynamic (instead of fixed length) and I'm trying to Marshal the bytes to a struct. I've read that the variable length char array needs to be IntPtr, but I haven't figured out how to Marshal it with the remaining bytes. I've also read in some articles that I might need a second structure, but still can't figure out how to Marshal it properly.

Here's one such site

What's the proper way to deal with variable length char arrays in structs?

The struct:

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct Header
{
    #region private member fields

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f1;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f2;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f3;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f4;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f5;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f6;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
    private char[] _f7;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 12)]
    private char[] _f8;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 32)]
    private char[] _f9;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f10;

    // how would this get filled with a char[] array from the byte array?
    public IntPtr VariableLengthData;

    #endregion
}

The function:

public static Header FromArray(byte[] array)
{
    IntPtr buff = IntPtr.Zero;

    try
    {
        int objsize = Marshal.SizeOf(typeof(Header));
        buff = Marshal.AllocHGlobal(objsize);
        Marshal.Copy(array, 0, buff, objsize);
        var result = (Header)Marshal.PtrToStructure(buff, typeof(HostHeader));

        // the variable length data needs to be filled in somehow
        // but also note that an extra 4 bytes was added to the size
        // of the struct with the IntPtr
        if(objsize < array.Length)
        {
            Marshal.Copy(array, array.Length - objsize, result.VariableLengthData, array.Length - objsize);
        }

        return result;
    }
    finally
    {
        if (buff != IntPtr.Zero)
        {
            Marshal.FreeHGlobal(buff);
            buff = IntPtr.Zero;
        }
    }
}

This works - but now Marshal.SizeOf(headerObj) says it's smaller than it really is when I attempt to convert it back to a byte[] array. Other than that, anything wrong with this solution?

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct Header
{
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 8)] 
    public char[] Field1;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 8)] 
    public char[] Field2;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 8)] 
    public char[] Field3;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 1)]
    public char[] Field4;
}

public static Header DeserializeHeader(byte[] data)
{
    int objsize = Marshal.SizeOf(typeof(Header));
    IntPtr buff = Marshal.AllocHGlobal(objsize);
    Marshal.Copy(data, 0, buff, objsize);
    var header = (Header)Marshal.PtrToStructure(buff, typeof(Header));
    Marshal.FreeHGlobal(buff);

    // resize Field4 to hold all the remaining bytes
    if(objsize < data.Length)
    {
        header.Field4 = Encoding.ASCII.GetChars(data, objsize - header.Field4.Length, data.Length - objsize - header.Field4.Length);
    }
    return header;
}

array+160 is the rest of your data. Encoding.ASCII.GetString(array,160,array.Length-160) — EZI
– EZI, Commented Feb 17, 2015 at 21:21
@EZI - that is correct. If the byte array is greater than 160, everything else is a variable length field. — Chris Gessler
– Chris Gessler, Commented Feb 17, 2015 at 21:30

Hans Passant · Accepted Answer · 2015-02-19 22:49:02Z

13

+200

Just don't try too hard to make your C# declarations match the packet format exactly. You'd favor this:

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 1)]
public class Header {
    //....
}

public class Packet {
    public Header Header;
    public byte[] VariableLengthData;
}

Now it becomes simple. You can marshal the header in one fell swoop and just copy whatever extra bytes are present:

static unsafe Packet FromArray(byte[] data) {
    var hdrsize = Marshal.SizeOf(typeof(Header));
    if (data.Length < hdrsize) throw new ArgumentException();
    Packet result = new Packet();
    // Marshal the header
    fixed (byte* pdata = &data[0]) {
        result.Header = (Header)Marshal.PtrToStructure(new IntPtr(pdata), typeof(Header)); 
    }
    // Copy the rest
    var varsize = data.Length - hdrsize;
    result.VariableLengthData = new byte[varsize];
    Array.Copy(data, hdrsize, result.VariableLengthData, 0, varsize);
    return result;
}

And modify the Packet class as you see fit, you probably want to add a bunch of properties to deal with the char[].

edited Feb 19, 2015 at 22:49

answered Feb 19, 2015 at 22:38

Hans Passant

946k151 gold badges1.8k silver badges2.6k bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Chris Gessler Over a year ago

I considered this solution as well, but also read an article that using "public fixed char Data[1]" also fixes this problem, but I don't have the complete source to determine how it was used or marshaled. I may end up going this route for safety purposes, but do you know what they mean by this in the following article? experts-exchange.com/Programming/Languages/C_Sharp/…

Chris Gessler Over a year ago

I took your idea and made it part of the original struct, which might not be the best solution (see question for example), and it seems to work for deserialization, but serializing it back to a byte[] array, I would need the actual size of the object and not just the sizeof(Header). Thoughts??

Hans Passant Over a year ago

I considered a one byte array but dismissed it. Your program can crash with an AVE if you ever get a packet with no extra data. Maybe you can get a guarantee that there always is some, I couldn't make the same assumption.

Chris Gessler Over a year ago

If I remember correctly, I DO get a null terminator with every response, so the size of the array will always be (known-data-len + 1) or (known-data-len + var-data-len + 1). With that, I should be able marshal the data correctly without issue AND take into consideration anything beyond known-data-len because if there isn't any additional data, the null terminator should fill up the extra byte. However, I still like your approach better, but will definitely cause me headaches trying to serialize it into XML or JSON.

Hans Passant Over a year ago

If you need to serialize then you have even more reason to make your C# declaration not the same as the wire format. Just keep Header out of it completely and declare Packet in whatever way you like it to be serialized.

Collectives™ on Stack Overflow

byte[] array to struct with variable length array

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related