1

I have this project where I want to get the JSON data from a file with array of json objects with repetitive attributes:

some-file: (each object is in 1 line, for simplicity I formatted it)

{
   "A":"ALL",
   "B":"3349256522",
   "Location":{

      "Country":"USA"
   },
   "EffectiveDate":"2020-03-04T14:15:52.063Z",
   "Demographic":{
      "Q":"done",
      "G":"ok",
      "AppVersion":"1.3.4",
   },
   "ApplicationId":"92398723892937",
   "Id":"23232993939333",
   "CreationDate":"2020-03-04T14:15:52.063Z"
}

{
   "A":"NONE",
   "B":"8469256522",
   "Location":{
      "Country":"SPAIN"
   },
   "EffectiveDate":"2020-03-04T14:15:52.063Z",
   "Demographic":{
      "Q":"done",
      "G":"ok",
      "AppVersion":"1.3.5",
   },

   "ApplicationId":"92398723892937",
   "Id":"23232993939333",
   "CreationDate":"2020-03-09T14:15:52.063Z"
}

{
   "A":"ALL",
   "B":"8469256522",
   "Location":{

      "Country":"USA"
   },
   "EffectiveDate":"2020-03-04T14:15:52.063Z",
   "Demographic":{
      "Q":"done",
      "G":"ok",
      "AppVersion":"1.3.4",
   },

   "ApplicationId":"92398723892937",
   "Id":"23232993939333",
   "CreationDate":"2020-03-11T14:15:52.063Z"
}

I would like to get some statistics for specific items, Country and AppVersion. How can I process it using NodeJs to produce this output:

{
   "stats":{
      "Country":{
         "USA":"2",
         "SPAIN":"1"
      },
      "AppVersion":{
         "1.3.4":"2",
         "1.3.5":"1"
      }
   }
}

Also, the input file 'some-file' is not a valid JSON file, any recommendation how to convert it to be valid (in code) ?

The original file is (1 line each object):
{   "A":"ALL",   "B":"3349256522",   "Location":{      "Country":"USA"   },   "EffectiveDate":"2020-03-04T14:15:52.063Z",   "Demographic":{      "Q":"done",      "G":"ok",      "AppVersion":"1.3.4"   },   "ApplicationId":"92398723892937",   "Id":"23232993939333",   "CreationDate":"2020-03-04T14:15:52.063Z"}
{   "A":"ALL",   "B":"3349256522",   "Location":{      "Country":"SPAIN"   },   "EffectiveDate":"2020-03-04T14:15:52.063Z",   "Demographic":{      "Q":"done",      "G":"ok",      "AppVersion":"1.3.5"   },   "ApplicationId":"92398723892937",   "Id":"23232993939333",   "CreationDate":"2020-03-04T14:15:52.063Z"}
{   "A":"ALL",   "B":"3349256522",   "Location":{      "Country":"ITALY"   },   "EffectiveDate":"2020-03-04T14:15:52.063Z",   "Demographic":{      "Q":"done",      "G":"ok",      "AppVersion":"1.3.4"   },   "ApplicationId":"92398723892937",   "Id":"23232993939333",   "CreationDate":"2020-03-04T14:15:52.063Z"}
2
  • Have you tried writing code yet? If so, can you share it? Thanks. Commented Apr 8, 2020 at 21:45
  • what is the file format? Commented Apr 8, 2020 at 22:40

3 Answers 3

1

You can use array.reduce to accumulate the count values:

let input = [{
   "A":"ALL",
   "B":"3349256522",
   "Location":{
      "Country":"USA"
   },
   "EffectiveDate":"2020-03-04T14:15:52.063Z",
   "Demographic":{
      "Q":"done",
      "G":"ok",
      "AppVersion":"1.3.4",
   },
   "ApplicationId":"92398723892937",
   "Id":"23232993939333",
   "CreationDate":"2020-03-04T14:15:52.063Z"
},
{
   "A":"NONE",
   "B":"8469256522",
   "Location":{
      "Country":"SPAIN"
   },
   "EffectiveDate":"2020-03-04T14:15:52.063Z",
   "Demographic":{
      "Q":"done",
      "G":"ok",
      "AppVersion":"1.3.5",
   },

   "ApplicationId":"92398723892937",
   "Id":"23232993939333",
   "CreationDate":"2020-03-09T14:15:52.063Z"
},
{
   "A":"ALL",
   "B":"8469256522",
   "Location":{

      "Country":"USA"
   },
   "EffectiveDate":"2020-03-04T14:15:52.063Z",
   "Demographic":{
      "Q":"done",
      "G":"ok",
      "AppVersion":"1.3.4",
   },

   "ApplicationId":"92398723892937",
   "Id":"23232993939333",
   "CreationDate":"2020-03-11T14:15:52.063Z"
}];

let result = input.reduce((acc,current) => {
    let c = current.Location.Country;
    let v = current.Demographic.AppVersion;
    if(acc.Country[c]){
        acc.Country[c]++;
    } else {
        acc.Country[c] = 1;
    }
    
    if(acc.AppVersion[v]){
        acc.AppVersion[v]++;
    } else {
        acc.AppVersion[v] = 1;
    }
    return acc;
}, {Country: {}, AppVersion: {}});

console.log({ stats: result });

Sign up to request clarification or add additional context in comments.

2 Comments

Thanks! also, the input file 'some-file' is not a valid JSON file, any recommendation how to convert it to be valid (in code) ?
@dngot_ I do see your original file now, doesn't seem to be difficult - all you need is square brackets (beginning and end) along with commas at the end of every line
1

First, your JSON is not valid and throws some errors so I fixed it a bit (did it very quick so the formatting is bad but at least Node can read it):

{
"data": [
{
   "A":"ALL",
   "B":"3349256522",
   "Location":{

      "Country":"USA"
   },
   "EffectiveDate":"2020-03-04T14:15:52.063Z",
   "Demographic":{
      "Q":"done",
      "G":"ok",
      "AppVersion":"1.3.4"
   },
   "ApplicationId":"92398723892937",
   "Id":"23232993939333",
   "CreationDate":"2020-03-04T14:15:52.063Z"
},

{
   "A":"NONE",
   "B":"8469256522",
   "Location":{
      "Country":"SPAIN"
   },
   "EffectiveDate":"2020-03-04T14:15:52.063Z",
   "Demographic":{
      "Q":"done",
      "G":"ok",
      "AppVersion":"1.3.5"
   },

   "ApplicationId":"92398723892937",
   "Id":"23232993939333",
   "CreationDate":"2020-03-09T14:15:52.063Z"
},

{
   "A":"ALL",
   "B":"8469256522",
   "Location":{

      "Country":"USA"
   },
   "EffectiveDate":"2020-03-04T14:15:52.063Z",
   "Demographic":{
      "Q":"done",
      "G":"ok",
      "AppVersion":"1.3.4"
   },

   "ApplicationId":"92398723892937",
   "Id":"23232993939333",
   "CreationDate":"2020-03-11T14:15:52.063Z"
}
]
}

Next, as for reading it in Node, I recommend fs-extra over the default fs module. Install with npm i fs-extra and in your code:

const fs = require('fs-extra');

// DON'T use Synchronous methods in production code. it is fine for learning however.
// I also named your json file "info.json"

let data = fs.readJsonSync('info.json');

let stats = {
    country: {},
    version: {}
};

// This iterates over each array element
data.data.forEach((entry) => {
    // let is similar to var, but not as global as var can be
    let country = entry.Location.Country;

    // First check if the country already exists in the stats. If not we can create it.
    // hasOwnProperty lets you check if an object already has a certain key
    if (!stats.country.hasOwnProperty(country)) stats.country[country] = 0;

    // Increase the counter on the country
    stats.country[country] += 1;

    // Repeat the above lines for demographic/appversion
});

console.log(stats); // returns: { country: { USA: 2, SPAIN: 1 }, version: {} }

I wrote this super quick so there may be issues. Let me know if it works for you!

2 Comments

Thanks! also, the input file 'some-file' is not a valid JSON file, any recommendation how to convert it to be valid (in code) ?
@dngot_ Not that I am aware of, but you can also use fs-extra to save JSON data in a "pretty" way: fs.writeJsonSync('stats.json', stats, { spaces: '\t' })). You can also replace the filename and variable name to re-save your 'some-file'. See fs-extra: writeJsonSync for more info. Also as I said in the answer, I highly suggest you learn about asynchronous functions. Luckily fs-extra makes it pretty easy. Edit: I should clarify, for fs-extra to write that data, it already needs to be valid.
1

Since your file is a collection of JSON documents back-to-back, all on one line, you can break it apart pretty simply. Something like:

const fs = require('fs-extra'); // to get async file functions. You can also use regular 'fs' with a callback

const yourFilePath = 'some/path.txt';

async function convertFileToArray(filepath){
  const rawContent = await fs.readFile(filepath,'utf8');

  // EDIT: The following is if there were *no* newlines in the file
  // const asJson = `[${ rawContent.replace(/\}\s*\{/g,'},{')}]`;

  // This is what you want if you nave newlines between each object:
  const asJson = `[${rawContent.split(/[\r\n]+/g).join(',')}]`;
  const asArray = JSON.parse(asJson);
  return asArray;
}

async function main(){
  const dataArray = await convertFileToArray(yourFilePath);
  // Process using something like the answers provided by others
}

Edit: Noticed after posting that you have newlines separating each object. Updated code to reflect that.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.