Monitor maximum memory consumption in Node.js process

Question

I'm looking for a cross-platform way to reliably monitor maximum memory consumption in Node.js process, regardless of whether there are leaks or not.

The processes in my case are both real applications and synthetic tests.

I would expect it to work like

process.on('exit', () => {
  console.log('Max memory consumption: ' + ...);
});

It was possible to trace memory consumption somehow with node --trace_gc ..., but this resulted in output that is hard to read (and probably hard to analyze programmatically). Also, GC didn't occur when a script ended too fast, even if RAM usage was substantial.

From what I have seen on the subject, usually memwatch is suggested for that, like:

require('memwatch-next').on('stats', stats => {
  console.log('Max memory consumption: ' + stats.max);
});

But in my case it triggered only when GC already happened or didn't trigger at all, so it was useless for determining RAM consumption peaks.

I would prefer to avoid GUI tools like node-inspector if possible.

Can this maximum memory consumption be reliably retrieved as a number from the application itself or CLI alone, cross-platform?

@EMX In fact, yes, thanks, it looks a thing that I need. pidusage seems to be crossplatform. I'll give it a try. — Estus Flask
– Estus Flask, Commented Aug 30, 2017 at 8:46

Jacob Horbulyk · Accepted Answer · 2019-04-10 17:08:07Z

23

You could use Node.js process.memoryUsage() method to get memory usage stat:

The process.memoryUsage() method returns an object describing the memory usage of the Node.js process measured in bytes.

It returns the object of the following format:

{
  rss: 4935680,       // Resident Set Size
  heapTotal: 1826816, // Total Size of the Heap
  heapUsed: 650472,   // Heap actually Used
  external: 49879     // memory usage of C++ objects bound to JavaScript objects managed by V8
}

To get the maximum memory consumption in Node.js process, process.nextTick method could be used. process.nextTick() method adds the callback to the next tick queue. Once the current turn of the event loop turn runs to completion, all callbacks currently in the next tick queue will be called.

let _maxMemoryConsumption = 0;
let _dtOfMaxMemoryConsumption;

process.nextTick(() => {
  let memUsage = process.memoryUsage();
  if (memUsage.rss > _maxMemoryConsumption) {
    _maxMemoryConsumption = memUsage.rss;
    _dtOfMaxMemoryConsumption = new Date();
  }
});

process.on('exit', () => {
  console.log(`Max memory consumption: ${_maxMemoryConsumption} at ${_dtOfMaxMemoryConsumption}`);
});

edited Apr 10, 2019 at 17:08

Jacob Horbulyk

2,6576 gold badges27 silver badges37 bronze badges

answered Aug 27, 2017 at 12:06

alexmac

19.7k7 gold badges64 silver badges74 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Estus Flask Over a year ago

Thanks, it works. There are typos in variable names. Also, it likely should run on same tick and on exit as well to get full coverage.

alexmac Over a year ago

@estus I fixed the variable names, thanks for mentioning this. What about adding calculation in exit callback, I don't think that it's necessary (of course if you don't want create a huge object there:)). This event usually used to cleanup memory.

Estus Flask Over a year ago

Yes, adding the calculation to exit in addition to nextTick works pretty well. But the problem I've encountered is that process.nextTick should be set up recursively in order to track it in async scripts, but the fact that there is incomplete interval prevents it from calling exit when script ends. I solved it with setInterval(calculate, 10).unref(); process.on('exit', calculate); process.on('exit', () => {...}).

C.Unbay Over a year ago

@alexmac do you know how to add npms to nodejs that are written in C&C++?

EMX · Accepted Answer · 2017-08-27 16:55:59Z

7

+200

If you try to benchmark the process from within itself, you will have distorted memory usage values. (if you want more information about this, just comment)

This is a little (crossplatform) tool I coded for checking the memory usage of another process, it spawns an independent process and watches every 100ms for memory usage in order to find the highest peak, outputs everytime a new peak is found and stops once child has ended.

It uses pidusage which is a crossplatform process ( cpu % and ) memory usage of a PID

Allows customization of the spawn (arguments to be passed along with the spawn) [could be updated for command line usage]

It will also work with any node binary name, since it will reuse the one used to start this tool.

'use strict'
const UI = {};  var ñ = "    "
const pusage = require('pidusage');
//:Setup the 'cmd' array to be the file and arguments to be used
const ANALYSIS = {cmd:['child.js']}
ANALYSIS.child = require('child_process').spawn(
 process.argv[0], // reuse to work with the same binary name used to run this (node|nodejs|...)
 ANALYSIS.cmd,   // array with filePath & arguments to spawn for this analisis
 { //So the child_process doesn't behave like a child
   detached:true,    
   stdio:['ignore'],
   env:null
 }
);
//:The Analysis
DoAnalysis(ANALYSIS.child.pid);
ANALYSIS.child.unref()
var memPeak = 0;
function PIDStat(){
 pusage.stat(ANALYSIS.pid, function(err, stat) {
  if(err){ CheckError(err) }else{
   if(stat.memory > memPeak){memPeak=stat.memory;PrintStat()}
   setTimeout(PIDStat,100); pusage.unmonitor(process.pid)
  }
 })
}
//:UI (just for display)
function DoAnalysis(PID){
 var s = '═'.repeat(ANALYSIS.cmd[0].toString().length)
 ANALYSIS.pid = PID;
 UI.top = '╒═'+s+'═╕'
 UI.mid = '│ '+ANALYSIS.cmd[0]+' │'
 UI.bot = '╘═'+s+'═╛'
 console.log(UI.x);
 PIDStat()
}
function PrintStat(){
 console.clear()
 console.log('\n',UI.top,'\n',UI.mid,'PEAK MEM. :',memPeak,'\n',UI.bot)
}
function CheckError(e){
 switch(e.code){
  case "ENOENT": console.log("              [the analysis ended]\n"); break;
  default: console.log("[/!\\ error]\n",e); break
 }
}

Will produce the following output :

 ╒══════════╕ 
 │ child.js │ PEAK MEM. : 28737536 
 ╘══════════╛
              [the analysis ended]

This tool prevents you from adding bloat to the code of the process you actually want to benchmark, so this way you wont get different memory usage values, since your bloat/benchmarking code will also add memory usage to that process.

answered Aug 27, 2017 at 16:55

EMX

6,2492 gold badges31 silver badges33 bronze badges

3 Comments

Estus Flask Over a year ago

The code above fails on console.clear() in Node 6, it was recently added to Node API. Thanks, it works as intended. It should be mentioned that it doesn't work well on synchronously running processes, the script can miss the point where they have a peak. As for the bloating argument, I found that process.memoryUsage() that runs on every tick and was suggested in the other answer provides zero impact on CPU and RAM, so it's not a big deal.

bluehipy Over a year ago

@estus How could a call have zero impact on CPU? :)

Estus Flask Over a year ago

@bluehipy It's negligible, not literally 0. I stand corrected, process.memoryUsage() gave 2-3% when monitoring on each tick and ~0% when monitoring with 10ms interval (which is totally acceptable in a situation like that)

Collectives™ on Stack Overflow

Monitor maximum memory consumption in Node.js process

2 Answers 2

4 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related