10

I've got an express app talking to mongodb via the node-mongodb-native driver. Some requests in my app are intermittently slow. Any good tools or strategies to confirm or rule out the driver connection pool size as a bottleneck?

Here's some discussion of tuning pool size, but it's pretty inconclusive. aheckmann notes that the default of 5 is usually plenty, while tinana saw significant gains from bumping up the pool with many concurrent request.

Update: This question was to help me understand tuning and tooling driver pool size rather than to troubleshoot the immediate performance issue. I described my issue only to give the question a little context.

Improve this question
4
  • 1
    Have you profiled? Are you sure that DB requests take most of the time? (Could use something simple like Puncher to figure that out. Check mongo.log out too.) Commented Aug 10, 2013 at 0:24
  • @elmigranto, some node profiling would definitely get me more info. DB communication seemed intuitively most smellly, so I was hoping to avoid setting up and deciphering gerenal profiling if good driver-specific tooling already exists. Commented Aug 12, 2013 at 5:51
  • Why would you want to spend time fixing "intuitively most smelly" things when you can get concrete data and get rid of actual problem? Commented Aug 12, 2013 at 6:47
  • Just exploring the tooling ecosystem. Commented Aug 14, 2013 at 1:32

1 Answer 1

Reset to default
8

In scenarios like this, the first step is always to start to with the DB.

If you have queries that are slow to respond, those queries should appear in the slow logs. Take a look at the official docs for profiling. The default value for "slow" queries is about 100ms, so if your slow queries are because of the DB, you will see evidence there.

Additionally, take a look at the graphs for the DB. By "the graphs", I mean your Nagios/Cacti/Zabbix/ServerDensity/MMS charts of what the server is doing. If you do not have these, start there. Tweaking the connection pool size is useless if you don't actually know how many connections you have or what your CPU looks like.

Any good tools or strategies to confirm or rule out the driver connection pool size as a bottleneck?

Once you have ruled out the DB and you have configured monitoring then you can open up the connection pool size. Once you have all of this in place. You will be able to tweak the pool size and ensure that you have (a) solved the problem and (b) not caused more problems.

The whole cycle is important.

If you muck with the connection pool but you're not watching the slow logs and the total connections then you'll just cause more problems.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Good points. We saw no slow queries coinciding with the slow requests. Mongo lock percentage (<1%), page faults (max 13/min), and connections (~50/6000) all looked healthy, which is why we began to suspect the node driver connection pool. Any thoughts on profiling the driver after the db has been ruled out?
Profiling the driver requires getting into the Node.JS debugging tools. Given that you're working in Node, these are probably handy to have around, but detailed profiling is not trivial in Node. That stated, you are sitting at 50 concurrent connections with MongoDB and you are tracking all of that information, so you have all of the tools you need to just open up the connection pooling and see if it improves.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.