Reflection is too slow while deserialising JSON strings into .NET objects

Question

I'm having some issues with System.Reflection in C#. I'm pulling data from a database and retrieving that data in a JSON string. I've made my own implementation of handling the data from JSON into my self declared objects using Reflection. However, since I ussually get a JSON string with an array of like 50 - 100 objects my program runs really slow because of the loops I'm using with reflection.

I've heard that reflection is slow but it shouldn't be this slow. I feel something is not right in my implementation since I have a different project where I use JSON.NET serializer and instantiate my objects a bit differently with reflection that runs just fine on the same output (less than a second) while my slow program takes about 10 seconds for 50 objects.

Below are my classses that I'm using to store data

class DC_Host
{
    public string name;

    public void printProperties()
    {
        //Prints all properties of a class usign reflection
        //Doesn't really matter, since I'm not usign this for processing
    }
}

class Host : DC_Host
{
    public string asset_tag;
    public string assigned;
    public string assigned_to;
    public string attributes;
    public bool? can_print;
    public string category;
    public bool? cd_rom;
    public int? cd_speed;
    public string change_control;
    public string chassis_type;
    //And some more properties (around 70 - 80 fields in total)

Below you'll find my methods for processing the information into the objects that are stored inside a List. The JSON data is stored inside a dictionairy that contains a another dictionairy for every array object defined in the JSON input. Deserialising the JSON happens in a matter of miliseconds so there shouldn't be a problem in there.

public List<DC_Host> readJSONTtoHost(ref Dictionary<string, dynamic> json)
{
    bool array = isContainer();

    List<DC_Host> hosts = new List<DC_Host>();

    //Do different processing on objects depending on table type (array/single)
    if (array)
    {
        foreach (Dictionary<string, dynamic> obj in json[json.First().Key])
        {
            hosts.Add(reflectToObject(obj));
        }
    }
    else
    {
        hosts.Add(reflectToObject(json[json.First().Key]));
    }

    return hosts;
}

private DC_Host reflectToObject(Dictionary<string,dynamic> obj)
{
    Host h = new Host();

    FieldInfo[] fields = h.GetType().GetFields();

    foreach (FieldInfo f in fields)
    {
        Object value = null;

        /* IF there are values that are not in the dictionairy or where wrong conversion is
         * utilised the values will not be processed and therefore not inserted into the 
         * host object or just ignored. On a later stage I might post specific error messages
         * in the Catch module. */

        /* TODO : Optimize and find out why this is soo slow */
        try
        {
            value = obj[convTable[f.Name]];
        }
        catch { }

        if (value == null)
        {
            f.SetValue(h, null);
            continue;
        }
        // Het systeem werkt met list containers, MAAAR dan mogen er geen losse values zijn dus dit hangt
        // zeer sterk af van de implementatie van Service Now.
        if (f.FieldType == typeof(List<int?>)) //Arrays voor strings,ints en bools dus nog definieren 
        {
            int count = obj[convTable[f.Name]].Count;
            List<int?> temp = new List<int?>();
            for (int i = 0; i < count; i++)
            {
                temp.Add(obj[convTable[f.Name]][i]);
                f.SetValue(h, temp);
            }
        }
        else if (f.FieldType == typeof(int?))
            f.SetValue(h, int.Parse((string)value));
        else if (f.FieldType == typeof(bool?))
            f.SetValue(h, bool.Parse((string)value));
        else
            f.SetValue(h, (string)value);
    }

    Console.WriteLine("Processed " + h.name);

    return h;
}

I'm not sure what JSON.NET's implementation is in the background for using reflection but I'm assumign they use something I'm missing for optimising their reflection.

To gain maximum performance one needs to generate new code to fill in the data. — CodesInChaos
– CodesInChaos, Commented Jul 27, 2012 at 7:48
But even without that, making good use of dictionaries and caching should give quite a speedup over your code. — CodesInChaos
– CodesInChaos, Commented Jul 27, 2012 at 7:50
You can emit new methods at runtime, which fill the data into the objects. So you only need reflection to build those methods (Reflection.Emit, Expression.Compile,...), but not to hydrate the objects. — CodesInChaos
– CodesInChaos, Commented Jul 27, 2012 at 7:59
@JoeyDewd, try running the slow version without attaching the debugger. You might be surprised by the performance. What kills your app is the many exceptions that occur. Handling exceptions is expensive, and much more so if the debugger is attached. Lesson learned: never use exceptions for common program flow control. — Lucero
– Lucero, Commented Jul 27, 2012 at 8:06

Community · Accepted Answer · 2017-05-23 12:00:32Z

3

Basically, high-performance code like this tends to use meta-programming extensively; lots of ILGenerator etc (or Expression / CodeDom if you find that scary). PetaPoco showed a similar example earlier today: prevent DynamicMethod VerificationException - operation could destabilize the runtime

You could also look at the code other serialization engines, such as protobuf-net, which has crazy amounts of meta-programming.

If you don't want to go quite that far, you could look at FastMember, which handles the crazy stuff for you, so you just have to worry about object/member-name/value.

edited May 23, 2017 at 12:00

CommunityBot

11 silver badge

answered Jul 27, 2012 at 8:02

Marc Gravell

1.1m273 gold badges2.6k silver badges3k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Joey Dewd Over a year ago

Yeah, I'm quite new to reflection to be honest and I know there are ways to program LowLevel with IL to achieve the best results and even create dynamic classes at runtime but for now I'm trying to avoid those paths since they are at the moment too difficult for me. I've fixed the problem by removing the Try Catch statement using TryGetvalue (although I have no idea why that seemeed to fix the performance issues). I'll look into FastMember though for future optimalisation issues :). Thank you

Joey Dewd · Accepted Answer · 2012-07-27 08:13:41Z

For people that are running into this article I'll post my solution to my problem in here. The issue wasn't really related to reflection. There are ways to improve the speed using Reflection like CodesInChaos and Marc Gravell mentioned where Marc even craeted a very usefull library (FastMember) for people with not too much experience in low level reflection.

The solution however was non related to reflection itself. I had a Try Catch statement to evaluate if values exist in my dictionary. Using try catch statements to handle program flow is not a good idea. Handling exceptions is heavy on performance and especially when you're running the debugger, Try Catch statements can drastically kill your performance.

//New implementation, use TryGetValue from Dictionary to check for excising values.
dynamic value = null;
obj.TryGetValue(convTable[f.Name], out value);

My program runs perfectly fine now since I omitted the TryCatch statement.

Collectives™ on Stack Overflow

Reflection is too slow while deserialising JSON strings into .NET objects

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related