Serializing an array of objects one at a time with JSON.NET

Question

I try to serialize multiple items to json, and formats them as an array.

It's event based, cause it's lot of data which could not be hold in memory as a whole collection. But I need to serialize each item into a file and format it like an array.

_jsonWriter = new JsonTextWriter(new StreamWriter("Output.json")));
DataGatherer.ItemGathered += item =>
{
    _jsonSerializer.Serialize(_jsonWriter, item);
    _jsonWriter.Flush();
};

Currently this outputs following:

{
  "Id": 218515,
  "Name": "A"
}{
  "Id": 118647,
  "Name": "B"
}

Since the serializer serializes each item as an object an does not know its an array.

So how can I tell the JSON.Net serializer to handle each item as an item of an array and formats the data like so:

[{ "Id": 218515, "Name": "A"},{"Id": 118647,"Name": "B"}]

Thanks for any hints!

Brian Rogers · Accepted Answer · 2014-06-21 03:11:16Z

6

I would create a simple JsonItemWriter class to wrap the JsonTextWriter. The class would just need to keep track of whether any items had been written to the output yet. If not, use the inner JsonTextWriter to write a StartArray to the stream before writing the item. When the outer ItemWriter is closed, write an EndArray to the stream. Then change your event handler to use the ItemWriter instead of the JsonTextWriter.

Here is what I had in mind for the JsonItemWriter:

class JsonItemWriter
{
    private JsonTextWriter innerWriter;
    private JsonSerializer serializer;

    public JsonItemWriter(JsonTextWriter innerWriter, JsonSerializer serializer)
    {
        this.innerWriter = innerWriter;
        this.serializer = serializer;
    }

    public void WriteItem(object item)
    {
        if (innerWriter.WriteState == Newtonsoft.Json.WriteState.Start)
        {
            innerWriter.WriteStartArray();
        }
        serializer.Serialize(innerWriter, item);
        innerWriter.Flush();
    }

    public void Close()
    {
        innerWriter.WriteEndArray();
        innerWriter.Close();
    }
}

Then set up your event handler like this:

_jsonWriter = new JsonTextWriter(new StreamWriter("Output.json"));
_itemWriter = new JsonItemWriter(_jsonWriter, _jsonSerializer);

DataGatherer.ItemGathered += item =>
{
    _itemWriter.WriteItem(item);
};

Here's a short demo using a mock DataGatherer to stand in for yours:

class Program
{
    static void Main(string[] args)
    {
        JsonSerializer jsonSerializer = new JsonSerializer();
        JsonTextWriter jsonWriter = new JsonTextWriter(new StreamWriter("Output.json"));
        JsonItemWriter itemWriter = new JsonItemWriter(jsonWriter, jsonSerializer);

        MockDataGatherer gatherer = new MockDataGatherer();
        gatherer.ItemGathered += item =>
        {
            itemWriter.WriteItem(item);
        };

        var items = new[]
        {
            new { Id = 218515, Name = "A" },
            new { Id = 118647, Name = "B" }
        };
        gatherer.SimulateReceivingItems(items);

        itemWriter.Close();

        using (StreamReader reader = new StreamReader("Output.json"))
        {
            Console.WriteLine(reader.ReadToEnd());
        }
    }
}

class MockDataGatherer
{
    public void SimulateReceivingItems(IEnumerable<object> items)
    {
        foreach (object item in items)
        {
            ItemGathered(item);
        }
    }

    public event ItemGatheredEventHandler ItemGathered;
    public delegate void ItemGatheredEventHandler(object item);
}

Here is the output of the above (notice the objects are now wrapped in an array):

[{"Id":218515,"Name":"A"},{"Id":118647,"Name":"B"}]

edited Jun 21, 2014 at 3:11

answered Jun 20, 2014 at 12:21

Brian Rogers

130k31 gold badges315 silver badges315 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

dasheddot Over a year ago

Thank you very much! Best elaborated answer and code sample I ever got :) Works great and is quite easy, although I thought Json.net provides built-in ways for doing this.

Brian Rogers Over a year ago

AFAIK, Json.Net only provides built-in ways for doing common tasks. If you could hold all the items in memory at once, you could simply add them to a list and serialize that when you are finished gathering. But once you get to the point of having too many items to fit in memory, you have to get creative and come up with a different approach (hence this solution). I'm glad it worked for you though!

artragis · Accepted Answer · 2014-06-20 11:23:32Z

2

Why don't you build an IEnumerable<YourObject>. It does not need to loead all the collection in your memory and when you give it to the serialiazer, it is parsed as an array. A simple way to do that would be to replace your ItemGathered event call by a simple yield return As you

answered Jun 20, 2014 at 11:23

artragis

3,7031 gold badge21 silver badges32 bronze badges

Collectives™ on Stack Overflow

Serializing an array of objects one at a time with JSON.NET

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related