JavaScript array not filtering duplicate items

Question

my array filter does not work and I not sure which part was wrong. My sample data:

var arr = [
    {accountID: '-KqIR-HT7orcPpe-lZa8', age: 31, gender: 'female'},
    {accountID: '-KqIR-GvEpHFiPFZRxbG', age: 59, gender: 'female'},
    {accountID: '-KqIR-GvEpHFiPFZRxbG', age: 59, gender: 'female'},
    {accountID: '-KqIR-GvEpHFiPFZRxbG', age: 59, gender: 'female'},
    {accountID: '-KqIR-GvEpHFiPFZRxbG', age: 59, gender: 'female'},
    {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female'},
    {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female'},
    {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female'},
    {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female'},
    {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female'},
    {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female}
];

I wanted to filter out the same accountID. I found this solution online:

arr = arr.filter( function( item, index, inputArray ) {
   return inputArray.indexOf(item) == index;
});

When I tried to print out the filtered array using this:

for(var i = 0; i < arr.length; i++){
        console.log(arr[i][0].accountID + ' ' + arr[i][0].age + ' ' + arr[i][0].gender);
}

I get back the exact same array with duplicate results. Which part was wrong? Thanks!

Are the objects inside arr the same objects (literally the same, not structurally the same) as inputArray? — MinusFour
– MinusFour, Commented Aug 10, 2017 at 3:38
It seems to me that your filter function doesn't actually do anything. The callback is called once per element of the original array; for each element, it also passes in the index of that element. The index of the element will always equal itself, so I would expect every item is returned by the filter. — user47589
– user47589, Commented Aug 10, 2017 at 3:40
@MinusFour Sorry but I thought the inputArray is just a parameter of the function? In this case, it is arr right? — QWERTY
– QWERTY, Commented Aug 10, 2017 at 3:41
@Amy Yeah it printed out the exact same array at the for loop that part — QWERTY
– QWERTY, Commented Aug 10, 2017 at 3:42
What is you condition that those elements are equal? That the accountID is the same or that all three fields are the same. You cannot find duplicates using inputArray.indexOf(item) == index because each of the objects you have in the array is an unique object, even if they have the same values. — t.niese
– t.niese, Commented Aug 10, 2017 at 3:45

MinusFour · Accepted Answer · 2017-08-10 03:58:27Z

2

Use findIndex instead, as these objects are all different.

var arr = [{
    accountID: '-KqIR-HT7orcPpe-lZa8',
    age: 31,
    gender: 'female'
  },
  {
    accountID: '-KqIR-GvEpHFiPFZRxbG',
    age: 59,
    gender: 'female'
  },
  {
    accountID: '-KqIR-GvEpHFiPFZRxbG',
    age: 59,
    gender: 'female'
  },
  {
    accountID: '-KqIR-GvEpHFiPFZRxbG',
    age: 59,
    gender: 'female'
  },
  {
    accountID: '-KqIR-GvEpHFiPFZRxbG',
    age: 59,
    gender: 'female'
  },
  {
    accountID: '-KqIR-EKbN02zAfCRyoe',
    age: 24,
    gender: 'female'
  },
  {
    accountID: '-KqIR-EKbN02zAfCRyoe',
    age: 24,
    gender: 'female'
  },
  {
    accountID: '-KqIR-EKbN02zAfCRyoe',
    age: 24,
    gender: 'female'
  },
  {
    accountID: '-KqIR-EKbN02zAfCRyoe',
    age: 24,
    gender: 'female'
  },
  {
    accountID: '-KqIR-EKbN02zAfCRyoe',
    age: 24,
    gender: 'female'
  },
  {
    accountID: '-KqIR-EKbN02zAfCRyoe',
    age: 24,
    gender: 'female'
  }
];
arr = arr.filter(function(item, index, inputArray) {
  return inputArray.findIndex(el => el.accountID === item.accountID) == index;
});

console.log(arr);

You could also use a Map for this:

var arr = [{
        accountID: '-KqIR-HT7orcPpe-lZa8',
        age: 31,
        gender: 'female'
      },
      {
        accountID: '-KqIR-GvEpHFiPFZRxbG',
        age: 59,
        gender: 'female'
      },
      {
        accountID: '-KqIR-GvEpHFiPFZRxbG',
        age: 59,
        gender: 'female'
      },
      {
        accountID: '-KqIR-GvEpHFiPFZRxbG',
        age: 59,
        gender: 'female'
      },
      {
        accountID: '-KqIR-GvEpHFiPFZRxbG',
        age: 59,
        gender: 'female'
      },
      {
        accountID: '-KqIR-EKbN02zAfCRyoe',
        age: 24,
        gender: 'female'
      },
      {
        accountID: '-KqIR-EKbN02zAfCRyoe',
        age: 24,
        gender: 'female'
      },
      {
        accountID: '-KqIR-EKbN02zAfCRyoe',
        age: 24,
        gender: 'female'
      },
      {
        accountID: '-KqIR-EKbN02zAfCRyoe',
        age: 24,
        gender: 'female'
      },
      {
        accountID: '-KqIR-EKbN02zAfCRyoe',
        age: 24,
        gender: 'female'
      },
      {
        accountID: '-KqIR-EKbN02zAfCRyoe',
        age: 24,
        gender: 'female'
      }
    ];
    let m = new Map(arr.map(obj => [obj.accountID, obj]));
    let noDuplicates = [...m.values()];
    
    console.log(noDuplicates);

edited Aug 10, 2017 at 3:58

answered Aug 10, 2017 at 3:44

MinusFour

14.5k4 gold badges32 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

QWERTY Over a year ago

But then when I used the for loop to print out the result, it is only printing out '-KqIR-HT7orcPpe-lZa8' with no duplicate. The other two data went missing.

MinusFour Over a year ago

@DeniseTan, how are you printing it out? The example here works fine it prints all 3 ids.

QWERTY Over a year ago

I used the console.log(arr) and as well as the for loop in question but it is only returning one result which is the one with no duplicate. The other with duplicates simply get filtered out]

MinusFour Over a year ago

There's no for loop in this example, when you run the snippet does it work for you?

QWERTY Over a year ago

Yeah the code snippet works but strangely when I place the code into the project, it is only returning one record.

Obsidian Age · Accepted Answer · 2017-08-10 03:48:10Z

1

In ES2015, you can use Set() to store unique values of any type.

The following example loops over each of the values in the array with .forEach(), and adds the unique ones to the secondary array uniq. Note that you'll need to both stringify and parse the objects in the array.

var arr = [{accountID: '-KqIR-HT7orcPpe-lZa8', age: 31, gender: 'female'}, {accountID: '-KqIR-GvEpHFiPFZRxbG', age: 59, gender: 'female'}, {accountID: '-KqIR-GvEpHFiPFZRxbG', age: 59, gender: 'female'}, {accountID: '-KqIR-GvEpHFiPFZRxbG', age: 59, gender:
'female'}, {accountID: '-KqIR-GvEpHFiPFZRxbG', age: 59, gender: 'female'}, {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female'}, {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female'}, {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender:
'female'}, {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female'}, {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female'}, {accountID: '-KqIR-EKbN02zAfCRyoe', age: 24, gender: 'female'}];

var uniq = new Set();
arr.forEach(e => uniq.add(JSON.stringify(e)));
var res = Array.from(uniq).map(e => JSON.parse(e));
console.log(res);

Hope this helps! :)

edited Aug 10, 2017 at 3:48

answered Aug 10, 2017 at 3:41

Obsidian Age

42.6k11 gold badges54 silver badges76 bronze badges

5 Comments

user47589 Over a year ago

love the avatar :)

Obsidian Age Over a year ago

@Amy -- I was thinking the same when I saw yours ;)

user47589 Over a year ago

friendship intensifies

QWERTY Over a year ago

@ObsidianAge Thanks so much for the explanation! I was about to ask you regarding the Set and you updated the solution with explanation!! Thanks so much!!

t.niese Over a year ago

Using JSON.stringify might work, but it is a risky "solution" there is no guarantee for the order of properties in a JavaScript object and in the JSON string, so you might store {"accountID": "-KqIR-HT7orcPpe-lZa8", "age": "31", "gender": "female"} and {"gender": "female", "accountID": "-KqIR-HT7orcPpe-lZa8", "age": "31"} in the set even though those should be handled as identical. @DeniseTan

Collectives™ on Stack Overflow

JavaScript array not filtering duplicate items

2 Answers 2

5 Comments

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related