C# exception thrown when parsing JSON String

Question

I'm studying C# and trying to get the code below to parse an incoming JSON recipe string that I convert to an XML document and I'm getting the following error

   at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
   at System.Text.Json.Utf8JsonReader.ConsumeStringAndValidate(ReadOnlySpan`1 data, Int32 idx)
   at System.Text.Json.Utf8JsonReader.ConsumeString()
   at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
   at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
   at System.Text.Json.Utf8JsonReader.Read()
   at System.Text.Json.JsonDocument.Parse(ReadOnlySpan`1 utf8JsonSpan, Utf8JsonReader reader, MetadataDb& database, StackRowStack& stack)
   at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedBytes)
   at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 json, JsonDocumentOptions options)
   at System.Text.Json.JsonDocument.Parse(String json, JsonDocumentOptions options)

I'm unsure as to why it's not working as the data I send to parse is a string. It works for simple strings however when sending a 'recipe' it throws the exception. The recipe is just a string that I create an XML document from (learning exercise). The recipe code is valid but it seems the JSON reader has issues parsing this string, I could be wrong in this. Here's the code:

TCP receive data method

 private async Task Listen()
        {
            try
            {
                while (true)
                {
                    Token.ThrowIfCancellationRequested();
                    TcpClient client = await server.AcceptTcpClientAsync();
                    Console.WriteLine("Connected!");
                    await Task.Run(async () => await HandleDevice(client), Token);
                }
            }
            catch (SocketException e)
            {
                Console.WriteLine("Exception: {0}", e);
            }
        }


private async Task HandleDevice(TcpClient client)
        {
            string imei = String.Empty;

            string data = null;
            Byte[] bytes = new Byte[80196];
            int i;


            try
            {
                using (stream = client.GetStream())
                {
                    while ((i = await stream.ReadAsync(bytes, 0, bytes.Length, Token)) != 0)
                    {
                        Token.ThrowIfCancellationRequested();
                        string hex = BitConverter.ToString(bytes);
                        data = Encoding.UTF8.GetString(bytes, 0, i);
                        Console.WriteLine(data);
                        processData(data);
                    }
                }
            }
            catch (OperationCanceledException) { }
            catch (Exception e)
            {
                Console.WriteLine("Exception: {0}", e.ToString());
            }
            finally
            {
                client.Close();
            }
        }

Process data method

public static void processData(string data)
{
    var jsonOptions = new JsonDocumentOptions
    {
        AllowTrailingCommas = true,
    };

    using (JsonDocument document = JsonDocument.Parse(data, jsonOptions))
    {

        JsonElement root = document.RootElement;

        // FOR DEBUGGING
        // foreach (JsonProperty element in root.EnumerateObject())
        //{
        //  Console.WriteLine($"{element.Name} ValueKind={element.Value.ValueKind} Value={element.Value}");
        //}

        // Parse the data response
        if (root.TryGetProperty("data", out JsonElement contentToParse))
        {

            if (string.IsNullOrEmpty(contentToParse.GetString()) == false)
            {

                // Data to parse
                if (contentToParse.GetString().Contains("RECIPE:"))
                {
                    string[] recipe = contentToParse.GetString().Split(':');
                    sendToFront(recipe[1]);
                }
                else
                {
                    Console.WriteLine("Something else to parse {0}", contentToParse.GetString());
                }

            }

        }

    }
}

My incoming JSON String looks like this: {"user": "me", "data": "RECIPE:<?xml version='1.0'..."} - The response is valid JSON and tested by online validators.

UPDATE: I've inspected the issue via console and it appears that only part of the recipe is received which could be causing the issue. How can I get the full JSON string before sending it to the processData method?

You haven't shown us the receive code. I'm a bit rusty with sockets but I'd guess you can keep testing whether there's more data ready and receiving until there isn't, perhaps allowing a few hundred ms for more data to arrive. Or put end of message marker in your protocol, and wait for that, etc. — Rup
– Rup, Commented Jul 29, 2022 at 9:33
@Rup - Sorry, updated the code above which includes the listen and handle methods — 832049823
– 832049823, Commented Jul 29, 2022 at 9:38
ReadAsync doesn't necessarily read a full JSON object, it might be more or less than one whole object. You need some kind of framing so that you know where one ends and the next begins. A common solution is to read a 4-byte size value from the stream, then read that many bytes. A StreamReader can do this quite neatly — Charlieface
– Charlieface, Commented Jul 29, 2022 at 10:36
@Charlieface - Yeah, I realise I could JSON decode an incoming object and then send it to XmlDocument but for this learning it is more about handling varying data types. — 832049823
– 832049823, Commented Jul 29, 2022 at 12:14

Charlieface · Accepted Answer · 2022-07-29 11:07:25Z

3

As mentioned by @StephenCleary, you need some kind of framing mechanism.

A simple one is to just prefix each string with its length when sending it. Then you read the length, and use StreamReader to read that many bytes.

private async Task HandleDevice(TcpClient client)
{
    Char[] length = new Char[2];
    Char[] data = new Char[80196];

    try
    {
        using (var stream = client.GetStream())
        using (var reader = new StreamReader(stream, Encoding.UTF8))
        {
            while ((i = await reader.ReadBlockAsync(bytes.AsMemory(), Token)) != 0)
            {
                var toRead = ((int)length[0]) | (((int)length[1]) << 16);
                var charsRead = await reader.ReadBlockAsync(data.AsMemory(0, toRead), Token);
                var str = new string(data, 0, charsRead);
                Console.WriteLine(str);
                if (charsRead != toRead)
                    throw new EndOfStreamException("Unexpected end of stream");

                processData(str);
                Array.Clear(data);
            }
        }
    }
    catch (OperationCanceledException) { }
    catch (Exception e)
    {
        Console.WriteLine("Exception: {0}", e.ToString());
    }
    finally
    {
        client.Close();
    }
}

There are more efficient implementations, but this should suffice for most purposes.

answered Jul 29, 2022 at 11:07

Charlieface

78.6k8 gold badges35 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

832049823 Over a year ago

How would this prefix work as the order of receiving isn't guaranteed. I could receive "user" part first or the "data" - unless I have that wrong?

Charlieface Over a year ago

The whole JSON string would be prefixed. Eg the string {"user": "me", "data": "RECIPE:<?xml version='1.0'..."} would be prefixed with 56

Stephen Cleary · Accepted Answer · 2022-07-29 10:44:31Z

2

There's nothing wrong with the JSON parser. It's getting an incomplete string from your socket code.

The best and easiest solution (by far) is to communicate at a higher abstraction; i.e., self-host ASP.NET Core instead of reinventing it using a custom TCP/IP protocol.

But if you must use a custom protocol, you'll need to design it correctly. I have a blog series that may help; in particular note the article on message framing, which is what is causing this particular problem. For a more complete guide, I have a video series on properly implementing a custom protocol using .NET - it is not for the faint of heart.

answered Jul 29, 2022 at 10:44

Stephen Cleary

460k78 gold badges721 silver badges858 bronze badges

1 Comment

832049823 Over a year ago

Thanks for the reference to your blog.

Collectives™ on Stack Overflow

C# exception thrown when parsing JSON String

2 Answers 2

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related