How to increase connection pool size for Mongodb using python

Question

I am using Mongodb as database for project and i am sending 1000 requests per sec to mongodb. I got a observation that the request which are front in queue works faster as compare to request which are last in queue. I am afraid if call to mongodb increase it will consume more time. I think this i can manage using pool connection.If yes let me know how we increased pool size using python.If no suggest me any other alternative

Following are some results to find documents with field snippetid which i had index

Some request which are send early

db.collection.find({'snippetid': '55a0466a414353801e4ff16a'})

Time Taken 0.00563097000122

db.collection.find({'snippetid': '559417cd5217d572fa120a21'})

Time Taken 0.00330901145935

Some request which are last in queue

db.collection.find({'snippetid': '55a4b1d45217d5e7609e19b2'})

Time Taken 1.95499396324

db.collection.find({'snippetid': '55a059d5be98fa09168b63f9'})

Time Taken 1.96221590042

score 9 · Accepted Answer · 2015-08-06 07:30:00Z

9

Hey you can increased pool size in python using pymongo. Syntax

from pymongo import MongoClient
client = MongoClient('host', port, maxPoolSize=200)

For more details check link

edited Aug 6, 2015 at 7:30

answered Aug 6, 2015 at 7:23

user3868300

Sign up to request clarification or add additional context in comments.

2 Comments

3ppps Over a year ago

thanx for your answer

Foo L Over a year ago

new link: pymongo.readthedocs.io/en/stable/faq.html

Blakes Seven · Accepted Answer · 2015-08-06 06:57:29Z

3

From the pymongo documentation for MongoClient:

maxPoolSize (optional): The maximum number of connections that the pool will open simultaneously. If this is set, operations will block if there are maxPoolSize outstanding connections from the pool. Defaults to 100. Cannot be 0.

And given the signature:

 pymongo.mongo_client.MongoClient(
     host='localhost', port=27017, document_class=dict,
     tz_aware=False, connect=True, **kwargs)

So adding maxPoolSize with your desired value within that call will set the size of the connection pool.

Note that "nothing is free" and this comes at a clear cost on both the client and server to maintain a large number of connections. With high request loads you might look at alternate processing methods, or at least balancing the load.

answered Aug 6, 2015 at 6:57

Blakes Seven

50.5k14 gold badges132 silver badges136 bronze badges

1 Comment

3ppps Over a year ago

thanx for your answer

Collectives™ on Stack Overflow

How to increase connection pool size for Mongodb using python

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related