mongodb - mongo MongoCursorNotFoundException in long-running query loop

Question

I have a simple query loop that gets a MongoCursorNotFoundException after processing about 44,000 of 96,945 documents in around 93 minutes.

MongoIterable<MasterDocument> query = masterCollection.find().noCursorTimeout(true);
    for (MasterDocument masterDocument : query) { ... do some stuff ... }

The "do some stuff" part takes a while, which is why the entire loop takes so long.

My problem is that I get this exception after handling maybe half of the documents in the collection.

I am running both the client application and the mongod server locally on my Windows 10 laptop, accessing the server via localhost.

The server log shows lots of messages like this:

{"t":{"$date":"2021-01-04T20:21:35.510-08:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn27","msg":"Slow query","attr":{"type":"command","ns":"master_database.MasterCollection","command":{"find":"MasterCollection","filter":{"hashCode":1753339282},"$db":"master_database","lsid":{"id":{"$uuid":"6a252f51-2c6e-4c01-ae03-1a80aab109e0"}}},"planSummary":"COLLSCAN","keysExamined":0,"docsExamined":96944,"cursorExhausted":true,"numYields":96,"nreturned":0,"queryHash":"DBC59907","planCacheKey":"DBC59907","reslen":121,"locks":{"ReplicationStateTransition":{"acquireCount":{"w":97}},"Global":{"acquireCount":{"r":97}},"Database":{"acquireCount":{"r":97}},"Collection":{"acquireCount":{"r":97}},"Mutex":{"acquireCount":{"r":1}}},"storage":{},"protocol":"op_msg","durationMillis":147}}

The last of these messages is followed by:

{"t":{"$date":"2021-01-04T20:21:35.521-08:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn27","msg":"Connection ended","attr":{"remote":"127.0.0.1:58990","connectionId":27,"connectionCount":14}}

{"t":{"$date":"2021-01-04T20:21:35.522-08:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn26","msg":"Connection ended","attr":{"remote":"127.0.0.1:58989","connectionId":26,"connectionCount":13}} {"t":{"$date":"2021-01-04T20:21:35.922-08:00"},"s":"I", "c":"-", "id":20883, "ctx":"conn25","msg":"Interrupted operation as its client disconnected","attr":{"opId":310196}}

I have tried:

• Using "noCursorTimeout(true)" on the query cursor (as shown above)
• Starting the server with "mongod --setParameter localLogicalSessionTimeoutMinutes=240". This last seems to have caused additional log messages that say "error":"Location13111: wrong type for field (expireAfterSeconds) long != int"

I am using mongod 4.4 and the latest mongo java api.

Answer 1

You may need to increase the default cursor idle timeout to bigger value in all shards and mongos:

check the parameter(default is 10 min = 600000 ms ):

use admin
db.runCommand({getParameter:1, cursorTimeoutMillis: 1})

and update to bigger value:

use admin
db.runCommand({setParameter:1, cursorTimeoutMillis: 600000000 })

also the COLSCAN in your logs indicate that you dont use indexes in your query , maybe you need to create one on "hashCode" ...