Javascript: Determine and return duplicates objects from array of objects based in multiple keys

Question

I have an array of objects:

    var data  = [{"monitor":"TFT",
                  "manufacturer":"MONCORP",
                  "monID":"1234",
                  "Delivery Way":"DELIVERY",
                  "BarCode Text":"Test",
                  "BarCode Id":"D9",
                  "Status":"OK"},               

                 {"monitor":"LCD",
                  "manufacturer":"MONCORP",
                  "monID":"",
                  "Delivery Way":"PICKUP",
                  "BarCode Text":"Dummy Text",
                  "BarCode Id":"P2",
                  "Status":"OK"},

                 {"monitor":"TFT",
                  "manufacturer":"MONCORP",
                  "monID":"1234",
                  "Delivery Way":"DELIVERY",
                  "BarCode Text":"ONLY TEST",
                  "BarCode Id":"D9",
                  "Status":"OK"}, 

                 {"monitor":"LCD",
                  "manufacturer":"MONCORP",
                  "monID":"1234",
                  "Delivery Way":"DELIVERY",
                  "BarCode Text":"FOR TESTING PURPOSE",
                  "BarCode Id":"D9",
                  "Status":"OK"},

                  {"monitor":"TFT",
                   "manufacturer":"MONCORP",
                   "monID":"",
                   "Delivery Way":"PICKUP",
                   "BarCode Text":"DUMMIEST TEXT",
                   "BarCode Id":"P7",
                   "Status":"OK"}];

So I want to take only the objects that are duplicated, BUT I want to distinguish them according to values of keys: monitor, manufacturer, monID, Delivery Way, BarCode Id, Status.

The expected result is:

    expected = [{"monitor":"TFT",
                 "manufacturer":"MONCORP",
                 "monID":"1234",
                 "Delivery Way":"DELIVERY",
                 "BarCode Text":"Test",
                 "BarCode Id":"D9",
                 "Status":"OK"},

                {"monitor":"TFT",
                 "manufacturer":"MONCORP",
                 "monID":"1234",
                 "Delivery Way":"DELIVERY",
                 "BarCode Text":"ONLY TEST",
                 "BarCode Id":"D9",
                 "Status":"OK"}]

by 'I want to distinguish them according to values of keys:..' do you mean you want to ignore the differences for 'BarCode Text'? (or any other key not in the list?) — Kaddath
– Kaddath, Commented Jun 29, 2017 at 9:01
Yes exactly i am only interested for the list: monitor, manufacturer, monID, Delivery Way, BarCode Id, Status. — Alex Botzis
– Alex Botzis, Commented Jun 29, 2017 at 9:03

emanek · Accepted Answer · 2017-06-29 11:33:27Z

1

It might be a good idea to do this on the database level (e.g. some sort of GROUP BY operation).

In JavaScript, you can iterate through the array and create a hash for each record, which should be made up of the fields that you want to be unique. This hash can be then used as a map key, i.e. inserting these record to a map with such key will eliminate the duplicates.

Example:

var map = {};
for (var i = 0; i < data.length; i++) {
    var key = data[i].monitor + "#" + data[i].monID + "#" + data[i].manufacturer + ... ;
    map[key] = data[i];
}

The map object will at the end contain only unique keys mapped to the last item with such key.

Note that the key is just a string concatenated of the fields you want to be unique. It could not work correctly if your fields contain the character #, or if they are not type string. If you want to go this way, I would suggest to compute a hash code from the string

To identify the duplicates, you can check in every step of the iteration whether the key is already in the map:

if (map[key]) {
   // This record is a duplicate
}

In order to group together the duplicate records, you can construct a map mapping key -> array of duplicates. This could be done like this (showing just the inside of the loop)

var key = ...
if (!map[key]) {
    // First time we see this key, let's add it to the map
    map[key] = [ data[i] ]; // Map the key to a new array containing the current record
} else {
    // Duplicate; just add this record to the existing 
    // list of records with the same key
    map[key].push(data[i]);
}

edited Jun 29, 2017 at 11:33

answered Jun 29, 2017 at 9:04

emanek

4212 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

Kaddath Over a year ago

this method works well and is commonly used. Don't forget to adapt it a little if the values in your data can be something else than strings. In case of objects or arrays, you can use JSON.stringify. In this case, you should use a hash code as suggested to avoid too long strings

Alex Botzis Over a year ago

Can we somehow get rid of these lines (these that i mentioned it expected result) ?

emanek Over a year ago

Sorry what do you mean by get rid off these lines...?

Alex Botzis Over a year ago

To reject those lines (actually to store them in a log file) who have in common values of keys: monitor, manufacturer, monID, Delivery Way, BarCode Id, Status.

Kaddath Over a year ago

@AlexBotzis yes, in the line beginning by var key = data[i].monitor. You explicitly specify here the fields you will use to compose your key. Just don't put here the fields you want to ignore

|

Collectives™ on Stack Overflow

Javascript: Determine and return duplicates objects from array of objects based in multiple keys

1 Answer 1

10 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related