We've recently upgraded our web application to MongoDB C# Driver 2.0 and deployed to production. Below a certain load, the application runs fine. Once the load on the production server exceeds a certain limit, the CPU of the application instantly falls down to 0 and after about 30 seconds, this exception is logged several times:
System.TimeoutException message: A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = ReadPreferenceServerSelector{ ReadPreference = { Mode = Primary, TagSets = System.Collections.Generic.List`1[MongoDB.Driver.TagSet] } }, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 } }. Client view of cluster state is { ClusterId : "1", Type : "Standalone", State : "Disconnected", Servers : [{ ServerId: "{ ClusterId : 1, EndPoint : "Unspecified/10.4.0.113:27017" }", EndPoint: "Unspecified/10.4.0.113:27017", State: "Disconnected", Type: "Unknown" }] }.
stack trace:
at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description)
at MongoDB.Driver.Core.Clusters.Cluster.<WaitForDescriptionChangedAsync>d__18.MoveNext()
--- End of stack trace
We are using a singleton MongoClient object, which is initiated like this:
private static object _syncRoot = new object();
private static MongoClient _client;
private static IMongoDatabase _database;
private IMongoDatabase GetDatabase()
{
...
if (_client == null)
{
lock (_syncRoot)
{
if (_client == null)
{
_client = new MongoClient(
new MongoClientSettings
{
Server = new MongoServerAddress(host, port),
Credentials = new[] { credentials },
});
_database = _client.GetDatabase("proddb");
return _database;
}
}
}
return _database;
}
public IMongoCollection<T> GetCollection<T>(string name)
{
return GetDatabase().GetCollection<T>(name);
}
A typical call to database looks like this:
public async Task<MongoItem> GetById(string id)
{
var collection = _connectionManager.GetCollection<MongoItem>("items");
var fdb = new FilterDefinitionBuilder<MongoItem>();
var f = fdb.Eq(mi => mi.Id, id);
return await collection.Find(f).FirstOrDefaultAsync();
}
How can we discover the reason and fix this issue?
