0

I am running mapreduce on a collection with 1.15M documents. In the case of duplicates, when I return a result, I want to prefer some documents values over others (i.e. a field value in one document may be more accurate than another's, but then again might not exist at all).

To choose a value, I define this function (arguments are in the order of most preferred to least preferred):

function getPreferredValue() {
var length = arguments.length;
for (var i = 0; i < length; i++) {
    if (arguments[i] != null) {
        return arguments[i];
    }
}
return null;
}

My mapreduce command looks like this:

db.runCommand({
mapreduce:"catalog",
query: {$or:[
    {name:/^HIP/i},
    {name:/^SAO/i}
]},
map:function() {
    emit({dummy:this.xref}, this); // give me the whole damn object then
},
reduce:function(key, values) {
    var tempObject = {hipObject: null, saoObject:null};
    for (var i = 0; i < values.length; i++) {
        if (values[i].name.search("/^HIP") != -1) {
            var hipObject = {bii: null, lii: null, class: null, radeg: null, decdeg: null, rapm:null, decpm:null, vmag:null, parallax:null, xref:null};
            hipObject.bii = values[i].bii;
            hipObject.lii = values[i].lii;
            hipObject.class = values[i].class;
            hipObject.radeg = values[i].radeg;
            hipObject.decdec = values[i].decdeg;
            hipObject.rapm = values[i].rapm;
            hipObject.decpm = values[i].decpm;
            hipObject.vmag = values[i].vmag;
            hipObject.parallax = values[i].parallax;
            hipObject.xref = values[i].xref;
            tempObject.hipObject = hipObject;
        }
        if (values[i].name.search("/^SAO") != -1) {
            var saoObject = {bii: null, lii: null, class: null, radeg: null, decdeg: null, rapm:null, decpm:null, vmag:null, parallax:null, xref:null};
            saoObject.bii = values[i].bii;
            saoObject.lii = values[i].lii;
            saoObject.class = values[i].class;
            saoObject.radeg = values[i].radeg;
            saoObject.decdec = values[i].decdeg;
            saoObject.rapm = values[i].rapm;
            saoObject.decpm = values[i].decpm;
            saoObject.vmag = values[i].vmag;
            saoObject.parallax = values[i].parallax;
            saoObject.xref = values[i].xref;
            tempObject.saoObject = saoObject;
        }
    }

    var result = {bii: null, lii: null, class: null, radeg: null, decdeg: null, rapm:null, decpm:null, vmag:null, parallax:null, xref:null}
    result.bii = getPreferredValue(tempObject.hipObject.bii, tempObject.saoObject.bii);
    result.lii = getPreferredValue(tempObject.hipObject.lii, tempObject.saoObject.lii);
    result.class = getPreferredValue(tempObject.hipObject.class, tempObject.saoObject.class);
    result.radeg = getPreferredValue(tempObject.hipObject.radeg, tempObject.saoObject.radeg);
    result.decdeg = getPreferredValue(tempObject.hipObject.decdeg, tempObject.saoObject.decdeg);
    result.rapm = getPreferredValue(tempObject.hipObject.rapm, tempObject.saoObject.rapm);
    result.decpm = getPreferredValue(tempObject.hipObject.decpm, tempObject.saoObject.decpm);
    result.vmag = getPreferredValue(tempObject.hipObject.vmag, tempObject.saoObject.vmag);
    result.parallax = getPreferredValue(tempObject.hipObject.parallax, tempObject.saoObject.parallax);
    result.xref = tempObject.hipObject.xref;

    return result;
},
out: {replace:"catalog2", db:"astro3"}
});

Unfortunately, when attempting to run, I get an error that getPreferredValue is not defined.

Probably a stupid error on my part, but can anyone assist? Jason

1
  • you should delete this question then. Commented Nov 30, 2011 at 18:40

2 Answers 2

1

You could also store the function in the map reduce command's 'scope' with

scope: {
    getPreferredValue: function(){...}
}
Sign up to request clarification or add additional context in comments.

Comments

0

Disregard, I wasn't saving the function server side.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.