Assign JSON value to variable based on value of a different key

Question

I have this function for extracting the timestamp from two JSON objects:

lineReader.on('line', function (line) {
    var obj = JSON.parse(line);
    if(obj.Event == "SparkListenerApplicationStart" || obj.Event == "SparkListenerApplicationEnd") {
        console.log('Line from file:', obj.Timestamp);
    }
});

The JSON comes from a log file(not JSON) where each line represents an entry in the log and each line also happens to be in JSON format on its own.

The two objects represent the start and finish of a job. These can be identified by the event key(SparkListenerApplicationStart and SparkListenerApplicationEnd). They also both contain a timestamp key. I want to subtract the end time from the start time to get the duration.

My thinking is to assign the timestamp from the JSON where Event key = SparkListenerApplicationStart to one variable and assign the timestamp from the JSON where Event key = SparkListenerApplicationEnd to another variable and subtract one from the other. How can I do this? I know I can't simply do anything like:

var startTime = if(obj.Event == "SparkListenerApplicationStart"){
    return obj.Timestamp;
}

If you are trying to measure the time between start and end, then we need to maintain a state outside the callback function of line. — Malice
– Malice, Commented Jan 8, 2018 at 14:07

pac · Accepted Answer · 2018-01-24 10:00:19Z

1

I'm not sure if I understood, but if are reading rows and want get the Timestamp of each row I would re-write a new object;

const collection = []

lineReader.on('line', function (line) {
    var obj = JSON.parse(line);
    if(obj.Event == "SparkListenerApplicationStart" || obj.Event == "SparkListenerApplicationEnd") {
        // console.log('Line from file:', obj.Timestamp);
        collection.push(obj.Timestamp)
    }
});

console.log(collection);

Where collection could be a LocalStorage, Global Variable, or something alike.

Additional info With regard to my comment where I queried how to identify the start and end times, I ended up setting start as the smallest value and end as the largest. Here is my final code:

const collection = []

lineReader.on('line', function (line) {
    var obj = JSON.parse(line);
    if((obj.Event == "SparkListenerApplicationStart" || obj.Event == "SparkListenerApplicationEnd")) {
        collection.push(obj.Timestamp);
        if(collection.length == 2){
            startTime = Math.min.apply(null, collection);
            finishTime = Math.max.apply(null, collection);
            duration = finishTime - startTime;
            console.log(duration);
        }
    }
});

edited Jan 24, 2018 at 10:00

pac

5112 gold badges9 silver badges31 bronze badges

answered Jan 8, 2018 at 14:05

Pablo

2,3315 gold badges34 silver badges65 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

pac Over a year ago

Thanks for the reply Pablo. I see whet you have done there and its good that I've got those two values but how do I then identify which is which and then subtract one from the other? I guess I could just find the smallest andd subtract it from the largest.

Pablo Over a year ago

You could work with binary tree, while you do the push.

Collectives™ on Stack Overflow

Assign JSON value to variable based on value of a different key

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related