find duplicate values in array of objects in javascript

Question

I have two lists in javascript that are of same structure like below:

var required_documents = [{"id":1,"dt":1},{"id":2,"dt":2},{"id":3,"dt":3}];
var existing_documents = [{"id":1,"dt":1},{"id":2,"dt":2},{"id":3,"dt":4}];

I need to remove all records from database that are in existing documents list (i.e "dt") but NOT in required_documents list. For the above scenario I should remove only {"id":3,"dt":4} and insert {"id":3,"dt":3}. I am not sure how I can compare on just one property. This is below that I found on SOF sometime ago but can't find it again apologies for not referencing it.

required_documents.forEach((obj) => {
    const elementInArr2 = existing_documents.find((o) => o.dt === obj.dt);
    console.log('found elementinarr: ' + obj.dt);
});

This returns unique objects like dt:1,dt:2,dt:3 but I need dt:4 from the existing documents list as it is the one that is not in the required documents list and needs to be deleted. How can I get just the one that is not in the required documents list.

What is the significant part of each record? Is the id relevant or are you only interested in dt? Could there possibly be other properties and if so, are they also significant? — Phil
– Phil, Commented Aug 5, 2022 at 4:35

Phil · Accepted Answer · 2022-08-05 05:20:21Z

1

Assuming both id and dt properties are significant, I would first create a means of hashing an entry and then build a hashed set of required_documents.

Then you can filter out anything from existing_documents that is in the set, leaving only the results you want.

const required_documents = [{"id":1,"dt":1},{"id":2,"dt":2},{"id":3,"dt":3}];
const existing_documents = [{"id":1,"dt":1},{"id":2,"dt":2},{"id":3,"dt":4}];

// a simple stringify hash
const createHash = ({ id, dt }) => JSON.stringify({ id, dt });

const requiredHashSet = new Set(required_documents.map(createHash));

const result = existing_documents.filter(
  (doc) => !requiredHashSet.has(createHash(doc))
);

console.log(result);

The hash creation can be anything that produces a comparable entity that can uniquely identify a record.

answered Aug 5, 2022 at 5:20

Phil

166k25 gold badges265 silver badges269 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Alaksandar Jesus Gene · Accepted Answer · 2022-08-05 05:01:23Z

1

You need to run it twice to confirm there is no elements left in existing. So create a function and use it.

var required_documents = [{"id":1,"dt":1},{"id":2,"dt":2},{"id":3,"dt":3}];
var existing_documents = [{"id":1,"dt":1},{"id":2,"dt":2},{"id":3,"dt":4}]

let output = [];
output = output.concat(extractUniqueValues(required_documents, output));
output = output.concat(extractUniqueValues(existing_documents, output));
console.log(output)
function extractUniqueValues(input, output){
  return input.filter((item)=>{
     return !output.find(v => v.dt == item.dt)
  })
}

answered Aug 5, 2022 at 5:01

Alaksandar Jesus Gene

6,89113 gold badges61 silver badges92 bronze badges

Comments

Madhav · Accepted Answer · 2022-08-05 05:24:49Z

1

You can do like below

var required_documents = [
  { id: 1, dt: 1 },
  { id: 2, dt: 2 },
  { id: 3, dt: 3 },
];
var existing_documents = [
  { id: 1, dt: 1 },
  { id: 2, dt: 2 },
  { id: 3, dt: 4 },
];

for (let index = 0; index < required_documents.length; index++) {
  const element = required_documents[index];
  for (var i = existing_documents.length - 1; i >= 0; i--) {
    const child = existing_documents[i];
    if (element.id === child.id && element.dt === child.dt) {
      existing_documents.splice(i, 1);
    } else {
      required_documents.push(element);
    }
  }
}

LOG  not exist [{"dt": 4, "id": 3}]
LOG unique items [{"dt": 1, "id": 1}, {"dt": 2, "id": 2}, {"dt": 3, "id": 3}]

answered Aug 5, 2022 at 5:24

Madhav

3512 silver badges14 bronze badges

Comments

Mike Szyndel · Accepted Answer · 2022-08-05 12:46:46Z

1

If you don't care about time complexity, something this should work:

var new_documents = existing_documents.filter(ed => {
  return required_documents.find(rd => rd.dt == ed.dt);
});

Edit Okay, I just reread your question and I'm a bit confused. Do you want the object {id: 3, dt: 3} inside the new array as well?

edited Aug 5, 2022 at 12:46

Mike Szyndel

10.6k11 gold badges51 silver badges64 bronze badges

answered Aug 5, 2022 at 4:29

olivia

131 silver badge4 bronze badges

Collectives™ on Stack Overflow

find duplicate values in array of objects in javascript

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related