Converting Non-Serializable Classes to a Byte Array

Question

I have a scenario where I am synchronizing data between multiple VERY dissimilar systems. (The data itself is similar but the tables on the different systems have quite different formats.) To assist with this synchronization, I have a database table which stores object hashes from each of the systems along with item keys and other relevant information. When the hash of an object from either system changes, I update the other.

My database table looks something like this.

CREATE TABLE [dbo].[SyncHashes](
    [SyncHashId] [int] IDENTITY(1,1) NOT NULL,
    [ObjectName] [nvarchar](50) NULL,
    [MappingTypeValue] [nvarchar](25) NULL,
    [MappingDirectionValue] [nvarchar](25) NULL,
    [SourceSystem] [nvarchar](50) NULL,
    [SourceKey] [nvarchar](200) NULL,
    [SourceHash] [nvarchar](50) NULL,
    [TargetSystem] [nvarchar](50) NULL,
    [TargetKey] [nvarchar](200) NULL,
    [TargetHash] [nvarchar](50) NULL,
    [UpdateNeededValue] [nvarchar](max) NULL,
    [CreatedOn] [datetime] NULL,
    [ModifiedOn] [datetime] NULL,
    [Version] [timestamp] NOT NULL, 
    [IsActive] [bit] NOT NULL,
PRIMARY KEY CLUSTERED 
(
    [SyncHashId] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

So far so good. But...

In order to effectively compute a hash (such as an MD5 hash (which is what i am using)) for an object, you need to be able to convert it to a byte array.

And...

It seems that in order to convert an object to a byte array, it must be serializable. (At least that's what I have read, and the errors I am getting from .NET seem to indicate that is true.)

For one of the systems, I have the ability to make all of my database objects serializable so that's great. Hashes get generated, everything gets synchronized, and the world is beautiful!

For another system, things are not so great. I am passed a database context from entity framework 4 (code first) model and the entities are NOT serialized.

When I attempt to cast as byte using something like the following, .NET complains and throws a minor tantrum--all the while refusing to give me the nice little byte array I so politely asked for.

foreach(var dataItem in context.TableName)
{
    var byteArray = (byte[]) dataItem;
}

Ok. No problem.

I have myself a nice little extension method which I thought might do the trick.

public static byte[] ObjectToByteArray<T>(this T obj)
{
    if (obj == null)
        return null;
    BinaryFormatter bf = new BinaryFormatter();
    MemoryStream ms = new MemoryStream();

    bf.Serialize(ms, obj);
    return ms.ToArray();
}

But oh no! If the object (the Entity) is not serializable, this routine throws me another nice little (and totally expected) exception.

So... I modify the routine and add a where clause to the method definition like so.

public static byte[] ObjectToByteArray<T>(this T obj) where T : ISerializable
{
    if (obj == null)
        return null;
    BinaryFormatter bf = new BinaryFormatter();
    MemoryStream ms = new MemoryStream();

    bf.Serialize(ms, obj);
    return ms.ToArray();
}

The only problem is that now I am back to square one where all of my objects need to be serializable to get a byte array.

Hmmm. Not good.

So I put together a hack to iterate through all of the object's properties and generate a string representation from which I could build a byte array. It was UGLY and INEFFICIENT but it kind of sort of did the trick.

public static string ComputeMD5Hash<T>(this T input)
{
    StringBuilder sb = new StringBuilder();

    Type t = input.GetType();
    PropertyInfo[] properties = t.GetProperties();

    foreach (var property in properties)
    {
        sb.Append(property.Name);
        sb.Append("|");
        object value = property.GetValue(input, null);
        if (value != null)
        {
            sb.Append(value);
        }
        sb.Append("|");
    }

    return MD5HashGenerator.GenerateKey(sb.ToString());
}

But...

After all that, what I still really would like to be able to is efficiently and properly to create a byte array from an object whose class is not marked as serializable. What is the best way to accomplish this?

Thank you in advance!

I forgot to mention that the MD5HashGenerator.GenerateKey(byte[] byteArray) function takes a byte array as its. parameter. — Anthony Gatlin
– Anthony Gatlin, Commented Oct 27, 2011 at 22:33

Community · Accepted Answer · 2020-06-20 09:12:55Z

9

create a byte array from an object whose class is not marked as serializable

You can use protobuf-net v2 to accomplish this. Download the zip then reference the protobuf-net assembly.

Consider this simple class definition we want to serialize:

public class Person
{
    public string Firstname { get; set; }
    public string Surname { get; set; }
    public int Age { get; set; }
}

You can then serialize this as a byte array:

var person = new Person {Firstname = "John", Surname = "Smith", Age = 30};
var model = ProtoBuf.Meta.TypeModel.Create();
//add all properties you want to serialize. 
//in this case we just loop over all the public properties of the class
//Order by name so the properties are in a predictable order
var properties = typeof (Person).GetProperties().Select(p => p.Name).OrderBy(name => name).ToArray();
model.Add(typeof(Person), true).Add(properties);

byte[] bytes;

using (var memoryStream = new MemoryStream())
{
    model.Serialize(memoryStream, person);
    bytes = memoryStream.GetBuffer();
}

The protobuf-net serializer will serialize much faster and produce a smaller byte[] array than BinaryFormatter

caveat 1 This will only (in its current form) serialize the public properties of your class, which looks ok for your usage.
caveat 2 This is considered brittle because adding a new property to Person may mean you are unable to deserialize a Person object that was serialized with the prior TypeModel.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Oct 27, 2011 at 23:05

wal

17.8k8 gold badges80 silver badges112 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Anthony Gatlin Over a year ago

Wal, this looks to be a FANTASTIC solution. However, I am currently working through one issue. The objects I am serializing are in a nested hierarchy and Protobuf seems to be having some difficulty with the child objects. (It does not seem to be automatically adding them to the model when it adds the parent.) Next, I will use reflection on the child objects to automatically add them to the model and then report back. All in all, I really like this Protobuf library. It seems blazingly fast. Thank you so much for sharing it.

wal Over a year ago

@Anthony Gatlin You will have to specify each type in the model you want serialized unfortunately. I've emailed the project creator (Marc Gravell) to find out if there is an existing way or if someone else has done it. Meanwhile, what you're suggesting by looping over the properties and adding each one to the model that is not a primitive sounds achievable.

wal Over a year ago

and Yes, Protobuf-net is very fast. Its used by this website under the hood.

Marc Gravell Over a year ago

Important: since GetProperties() does not guarantee the order, you should ensure that properties represents a repeatable order. The simplest approach would be to just use either .Select(p => p.Name).OrderBy(x => x).ToArray(), or Array.Sort(properties); (after the LINQ).

Anthony Gatlin Over a year ago

@Marc, Just out of curiosity, why does the property order matter? Does the serializer use property position rather than name when performing the transformation?

|

Collectives™ on Stack Overflow

Converting Non-Serializable Classes to a Byte Array

1 Answer 1

8 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related