Parsing non-standard JSON in JavaScript

Question

I am receiving a non-standard JSON format from a request. How one can convert or parse this kind of non-standard format in JavaScript or any other scripting language?

{
    "count": 2,
    "sales": [
        {
            "id": 1195816,
            "city": "New York",
            "name": "testing1"
        },
        {
            "id": 1195815,
            "city": "LA",
            "name": "testing2"
        }
    ],
    "total_sales_count": 148393,
    "date": "17.04.2020"
}
{
    "count": 2,
    "sales": [
        {
            "id": 1195816,
            "city": "Washington",
            "name": "testing3"
        },
        {
            "id": 1195815,
            "city": "New Jersey",
            "name": "testing4"
        }
    ],
    "total_sales_count": 49403,
    "date": "17.04.2020"
}

@AnuragSrivastava The one that you can validate. Try this: jsonlint.com — thatthing
– thatthing, Commented Apr 18, 2020 at 17:36
@sensahin Did you format these objects? Surely each object itself is on its own line... otherwise, you'll have to use a streaming parser after all. — Brad
– Brad, Commented Apr 18, 2020 at 17:41
Ths is called a New-Line Delimited JSON stackoverflow.com/questions/36122391/… — Grogu
– Grogu, Commented Mar 30, 2022 at 21:27

AKX · Accepted Answer · 2020-04-18 17:38:52Z

3

So you have multiple JSON objects concatenated, separated with a newline.

A quick hack that might just work is to add a comma between the objects and wrap them in an array.

> JSON.parse(`[${json.replace(/\}\n\{/g, '},{')}]`);
(2) [{…}, {…}]

answered Apr 18, 2020 at 17:38

AKX

171k16 gold badges146 silver badges228 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Brad · Accepted Answer · 2020-04-18 17:40:20Z

0

I am receiving a non-standard JSON...

In fact, there is a standard. Line-delimited, or ND-JSON. http://ndjson.org/

It's desirable to use this format when returning multiple records. It allows you to stream the response, and parse without having to load the entire dataset into memory at the same time.

To parse in JavaScript, I'd pick up one of the several standard packages available for this. For example: https://www.npmjs.com/package/ndjson

fs.createReadStream('data.txt')
  .pipe(ndjson.parse())
  .on('data', function(obj) {
    // obj is a javascript object
  })

If you want to do this in a browser that supports transform streams, here's some code from one of my projects you can adapt:

return res.body
    // Decode as UTF-8 Text
    .pipeThrough(new TextDecoderStream())

    // Split on lines
    .pipeThrough(new TransformStream({
      transform(chunk, controller) {
        textBuffer += chunk;
        const lines = textBuffer.split('\n');
        for (const line of lines.slice(0, -1)) {
          controller.enqueue(line);
        }
        textBuffer = lines.slice(-1)[0];
      },
      flush(controller) {
        if (textBuffer) {
          controller.enqueue(textBuffer);
        }
      }
    }))

    // Parse JSON objects
    .pipeThrough(new TransformStream({
      transform(line, controller) {
        if (line) {
          controller.enqueue(JSON.parse(line));
        }
      }
    }));

edited Apr 18, 2020 at 17:40

answered Apr 18, 2020 at 17:37

Brad

164k57 gold badges380 silver badges559 bronze badges

2 Comments

AKX Over a year ago

This might not work since OP's data isn't strictly one-JSON-object-per-line (but one-object-per-many-lines).

Brad Over a year ago

@AKX Ah, I assumed he formatted them for the question. Good point.

Collectives™ on Stack Overflow

Parsing non-standard JSON in JavaScript

2 Answers 2

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related