checking duplicates in javascripts and/or angularjs

Question

I am iterating through .csv file to do some element level validation in angular js app. I couldn't find better library in angular so i started writing custom javascript to handle this scenario. Below is the attached data sample ( interested in first column CN and 5th column NAME data ). All i could think of is few if conditions to check the index of i, j and store the values. Any suggestions would be appreciated.

CN  N   ACTIVE  TYPE    NAME       NO   COM
USA 45  Engin   Fed     Anderson   #10  NA
USA 46  Sports  BB      Kobe       #1   NA
USA 32  Sports  Soccer  Kobe       #17  NA
GER 30  Sports  Soccer  Gotze      #12  NA
GER 27  Sports  Soccer  Ozil       #18  NA
ITA 38  Sports  Soccer  Buffon     #2   NA

code snippet

for ( var i=0; i< file.length; i++ ){
    var singleRowData = file[i].split(',');
    singleRowData = singleRowData.filter(function(n){ return n != "" });
    for ( var j=0; j< singleRowData.length; j++){
        duplicateFunction(singleRowData, singleRowData[j], j, i);
    } 
}  

function duplicateFunction ( singleRowData, singleRowDataElement, singleRowDataElementIndex,  singleRowDataIndex){
/*
  handle the duplicates
*/
}

If I find same values in CN column for consecutive rows then i would like to check if i have duplicate values in NAME column for those rows.
If i don't have duplicate NAME for different rows of same CN values ( different rows ) then i shouldn't throw error.

In this data example, I should catch exception on 3rd row for CN = USA, NAME=Kobe and rest of the data should work fine

trincot · Accepted Answer · 2017-03-21 22:44:06Z

1

You could store the key pairs (CN + NAME) as a concatenated key in a keys object, and when you find it there for a new record, you know you have duplicate:

var file = [
    'USA,45,Engin,Fed,Anderson,#10,NA',
    'USA,46,Sports,BB,Kobe,#1,NA',
    'USA,32,Sports,Soccer,Kobe,#17,NA',
    'GER,30,Sports,Soccer,Gotze,#12,NA',
    'GER,27,Sports,Soccer,Ozil,#18,NA',
    'ITA,38,Sports,Soccer,Buffon,#2,NA'
];

var keys = {}; // store the keys that you have processed
var result = []; // array with accepted rows
for ( var i=0; i< file.length; i++ ){
    var singleRowData = file[i].split(',');
    singleRowData = singleRowData.filter(function(n){ return n != "" });
    var key = singleRowData[0] + '|' + singleRowData[4]; // CN + NAME
    if (key in keys) {
        console.log("Duplicate at " + i + " is ignored");
    } else {
        keys[key] = 1; // register key
        result.push(singleRowData);
    }        
}  
console.log(result);

.as-console-wrapper { max-height: 100% !important; top: 0; }

Here is a more compact ES6 version of the same idea, but with a ES6 Map and reduce:

const file = [
    'USA,45,Engin,Fed,Anderson,#10,NA',
    'USA,46,Sports,BB,Kobe,#1,NA',
    'USA,32,Sports,Soccer,Kobe,#17,NA',
    'GER,30,Sports,Soccer,Gotze,#12,NA',
    'GER,27,Sports,Soccer,Ozil,#18,NA',
    'ITA,38,Sports,Soccer,Buffon,#2,NA'
];

const result = [...file.reduce( (result, line) => {
    const singleRowData = line.split(',').filter(n => n != ""),
          key = singleRowData[0] + '|' + singleRowData[4]; // CN + NAME
    return result.has(key) ? result : result.set(key, singleRowData);
}, new Map).values()];
console.log(result);

.as-console-wrapper { max-height: 100% !important; top: 0; }

edited Mar 21, 2017 at 22:44

answered Mar 21, 2017 at 22:37

trincot

357k38 gold badges282 silver badges338 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Mad-D Over a year ago

Thank you, How does keys empty object has values of key without any assignment ? if (key in keys) - ?

trincot Over a year ago

It is in the comments of the code: at keys[key] = 1;. The value 1 is not important, just that the key is created.

Mad-D Over a year ago

thanks, make sense. So the value can be anything ? how does this help in assigning a key to keys object ? without the value it doesn't assign.

Collectives™ on Stack Overflow

checking duplicates in javascripts and/or angularjs

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related