Java: Best way to compare all keys within array of json objects?

Question

I'm parsing json files (size varies from few KB to few GB) that are built as an array of objects, for example:

{  
  "records": [
    {
       "col1": "someValue",
       "col2": "someValue",
       "col3": "someValue", 
    },
    {
       "col1": "someValue",
       "col2": "someValue",
       "col3": "someValue",
    },
    {
       "col1": "someValue",
       "col2": "someValue",
       "col3": "someValue",
    }  
  ]
}

The records represent individual rows of data from a table and the file always contains data for one table only.

I can extract table's metadata and parse it without any issues. I'm using JSON.simple library to do this.

What I'm trying to do now, is to validate that all of the objects have the same keys, no more or less as data needs to be ingested into a table. I can extract the keys using keySet() method and put it into list but it seems like comparing one list to an another times amount of rows (from few to millions) is a very poor and costly implementation.

Is there some nice solution that could quickly compare all keys from all json objects in json array?

Do you know the required column names beforehand?

Quant
– Quant

2016-09-30 11:31:30 +00:00
Commented Sep 30, 2016 at 11:31 — Quant
– Quant, Commented Sep 30, 2016 at 11:31
No, I don't know column names.

m4r74
– m4r74

2016-09-30 11:32:06 +00:00
Commented Sep 30, 2016 at 11:32 — m4r74
– m4r74, Commented Sep 30, 2016 at 11:32

Nicolai Ehemann · Accepted Answer · 2016-09-30 13:01:47Z

1

You can not avoid having to look at every key (amount n) of every row (amount m) of your data, so the complexity can not go below O(n * m), anyways.

answered Sep 30, 2016 at 13:01

Nicolai Ehemann

5744 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

m4r74 Over a year ago

Yeah, sure, I do need to look at every key in every object, but what will be the best possible approach? I was hoping that there would be a method in one of the json libraries that would allow to validate keys..?

m4r74 Over a year ago

or maybe not validate but compare as validation would require a schema.

Collectives™ on Stack Overflow

Java: Best way to compare all keys within array of json objects?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related