System.OutOfMemoryException with JSON.NET with List<object>

Question

I'm doing a process where my server generates a file with JSON.Net with 25000 records and size of 85MB, then this file is imported by client.

The client downloads the file and makes the deserialize with JSON.Net.

My problem is that the times and on different machines, deserialize command is giving:

System.OutOfMemoryException

Note: I tried to manually use the GC did not work.

Code generate error:

var listAddress = JsonConvert.DeserializeObject<List<address>>(File.ReadAllText(@"c:\Temp\test.json");

Code generate JSON File:

using (StreamWriter file = File.CreateText("C:\\test.json"))
{
    JsonSerializer serializer = new JsonSerializer();                        
    serializer.Serialize(file, listAddress);                        
}

My Class Address

public class Endereco
{
    public int Codigo { get; set; }
    public string CGCCPF { get; set; }
    public char? TipoPessoa { get; set; }
    public string UF { get; set; }
    public string SiglaDoRG { get; set; }
    public string MaeFantasica { get; set; }
    public string DescEndereco { get; set; }
    public string Complemento { get; set; }
    public string Bairro { get; set; }
    public string CEP { get; set; }
    public string Pai { get; set; }
    public string Cidade { get; set; }
    public string Telefone { get; set; }
    public string Fax { get; set; }
    public string Email { get; set; }
    public string IEouRG { get; set; }
    public string OrgaoEmissorRG { get; set; }
    public DateTime? DataNascimento { get; set; }
    public CampoObservacao Observacao { get; set; }
    public DateTime? DataRegistro { get; set; }
    public DateTime? DataUltAlteracao { get; set; }
    public int? CodigoFuncionario { get; set; }
    public string Ramal { get; set; }
    public DateTime? DataDeEmissaoRG { get; set; }
    public string Alfa1 { get; set; }
    public string Alfa2 { get; set; }
    public double? Num1 { get; set; }
    public double? Num2 { get; set; }
    public string PontoReferencia { get; set; }
    public string Celular { get; set; }
    public string EnderecoWeb { get; set; }
    public string Conta { get; set; }
    public string Rasocial { get; set; }
    public int? NcCodigo { get; set; }
    public int? NcCodigoC { get; set; }
    public string CFOP { get; set; }
    public string Numero { get; set; }
    public int? CodigoMunicipio { get; set; }
    public int? CodigoPais { get; set; }
    public string Suframa { get; set; }
    public string NumeroNit { get; set; }
    public string InscricaoMunicipal { get; set; }
    public string IE { get; set; }

    public bool Ativo
    {
        get { return !Observacao["Status"].Equals("I"); }
        set { Observacao["Status"] = !value ? "I" : String.Empty; }
    }

    public Endereco()
    {
        Observacao = new CampoObservacao();
    }

    public char? TipoDeMercado
    {
        get { return !String.IsNullOrEmpty(Observacao["Tipo de Mercado"]) ? (char?)Observacao["Tipo de Mercado"][0] : null; }
        set { Observacao["Tipo de Mercado"] = value.HasValue ? value.ToString() : String.Empty; }
    }

    public char? EstadoCivil
    {
        get { return !String.IsNullOrEmpty(Observacao["EstadoCivil"]) ? (char?)Observacao["EstadoCivil"][0] : null; }
        set { Observacao["EstadoCivil"] = value.HasValue ? value.ToString() : String.Empty; }
    }

    public Cliente Cliente { get; set; }
    public Funcionario Funcionario { get; set; }
    public Filial Filial { get; set; }
}

possible duplicate of Deserialize json array stream one item at a time — Brian Rogers
– Brian Rogers, Commented Dec 5, 2014 at 21:23

dbc · Accepted Answer · 2018-02-01 22:43:40Z

I was playing around a bit with the answer linked above: Deserialize json array stream one item at a time. The following slight modification might meet your needs. It reads and deserializes the JSON item-by-item rather than loading the entire list into memory:

public static class JsonConvertExtensions
{
    public static IEnumerable<T> DeserializeEnumerableFile<T>(string filename)
    {
        using (var stream = new StreamReader(filename))
            foreach (var item in DeserializeEnumerable<T>(stream))
                yield return item;
    }

    public static IEnumerable<T> DeserializeEnumerableString<T>(string json)
    {
        using (var sr = new StringReader(json))
            foreach (var item in DeserializeEnumerable<T>(sr))
                yield return item;
    }

    public static IEnumerable<T> DeserializeEnumerable<T>(TextReader textReader)
    {
        var serializer = JsonSerializer.CreateDefault();

        using (JsonTextReader reader = new JsonTextReader(textReader))
        {
            while (reader.Read())
            {
                if (reader.TokenType == JsonToken.StartObject)
                {
                    // Load each object from the stream and do something with it
                    yield return serializer.Deserialize<T>(reader);
                }
            }
        }
    }
}

Collectives™ on Stack Overflow

System.OutOfMemoryException with JSON.NET with List<object>

Code generate error:

Code generate JSON File:

My Class Address

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Code generate error:

Code generate JSON File:

My Class Address

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related