This is part 2/6 of the MongoDB sharded cluster write path series.
Questions this page covers: Does a write that hit a socket timeout stop on the server? If the driver's retryWrites and the application's retry are both on, how many times does one write arrive at the server? How many run on the server at the same time? How much of this overlap does maxTimeMS prevent?
Reference version: MongoDB 4.2 (source tagr4.2.12, 4.2 official documentation), MongoDB Java driverr5.4.0, retryable writes and driver sessions specifications (as fetched on 2026-11-05). The behavior of the latest server versions was not checked.
Write retries happen separately in two layers. When a network error occurs, the driver sends the same write once more under the retryable writes rules. The application (for example, a Spring Kafka consumer) catches the exception the driver finally throws and runs the write again from the start. The two counts multiply.
socketTimeoutMS is how long the client waits for a response before giving up. The server does not know about this timeout. In MongoDB 4.2, insert, update, delete, and findAndModify keep running on the server even after the client connection is closed.
The driver's resend carries the same session id (lsid) and transaction number (txnNumber) as the original request. The server treats the two as the same write and runs it only once. An application retry is a new write with a new session or a new number. On this page, each separate execution of the same write in the server engine is called a copy. Sends to the server reach at most twice the number of application executions. Overlapping copies reach at most the number of application executions.
Both settings are time limits, but a different side holds the clock.
| Item | socketTimeoutMS |
maxTimeMS |
|---|---|---|
| Who holds the clock | Client (driver) | Server |
| When time runs out | The driver gives up reading the socket and throws an exception | The server stops the operation at the next interrupt check point |
| The server operation | Does not know. Keeps running | Ends with a MaxTimeMSExpired error |
| Where it is set | Connection string, MongoClientSettings |
An argument attached to each command |
The 4.2 documentation "Terminate Running Operations" describes maxTimeMS like this.
When the operation reaches the specified time limit, MongoDB interrupts the operation at the next interrupt point.
An interrupt point is a code location where the operation checks for itself whether "there is a mark telling me to stop". The operation stops only when it reaches such a location. This style is called cooperative interruption. Part 4/6 covers it in detail.
Regardless of the command type, the server reads the generic argument maxTimeMS and sets a deadline on the operation (src/mongo/db/service_entry_point_common.cpp:817-830 (r4.2.12)). So attaching maxTimeMS to an insert command also sets a deadline.
Starting with MongoDB 4.2, some commands receive a kill mark (the same mark as killOp) when the client disconnects. The list in the "Client Disconnection" section of the 4.2 release notes has 10 commands. They are read commands such as find, aggregate (without $out or $merge), count, and distinct, plus authentication and listing commands.
insert, update, delete, and findAndModify are not on the list. In the source, too, the only files that call handleMarkKillOnClientDisconnect, the function that sets the kill mark on disconnect, are the read command files above. The write command file write_commands.cpp does not call this function.
Reads that receive the kill mark also stop cooperatively. If the operation cannot reach an interrupt check point, the read keeps running too.
The first sentence of the official documentation "Retryable Writes" sets the scope of retryable writes.
Retryable writes let drivers retry specific write operations once after network errors or if they cannot find a healthy primary in the replica set or sharded cluster.
Drivers compatible with MongoDB 4.2 and later turn this feature on by default. To turn it off, put retryWrites=false in the connection string. Retries apply to writes that change only one document.
| Category | Examples |
|---|---|
| Retried | insertOne, insertMany, updateOne, replaceOne, deleteOne, findOneAndUpdate, etc. |
| Retried | bulkWrite containing only the single-document writes above (both ordered and unordered) |
| Not retried | updateMany, deleteMany, a bulk containing even one multi: true update or limit: 0 delete |
| Not retried | Writes with write concern {w: 0} |
A socket read timeout is also a retryable error. In Java driver r5.4.0, MongoSocketReadTimeoutException extends MongoSocketException. CommandOperationHelper.isRetryableException returns true for a MongoSocketException. The retryable writes specification also says to attach the RetryableWriteError label to network errors.
The retryable writes specification sets the resend rule like this.
When retrying a write command, drivers MUST resend the command with the same transaction ID.
A transaction ID is the pair of lsid (server session id) and txnNumber (a number that grows within that session). When mongod receives a request with a txnNumber, it first checks out that session. Checking out means reserving the session so that only one operation uses it. The comment at service_entry_point_common.cpp:397-399 (r4.2.12) says "only requests with a transaction number will check out the session".
Since the original request and the resend compete for the same session, the result is one of two.
Either way, only one copy actually performs the write. The resend reaches the server but creates no extra work.
If the driver's resend also fails, the exception goes up to the application. When the application runs the same write again, the driver treats it as a write it has never seen.
lsid.txnNumber goes up.So if the original operation is still running, the two operations run side by side inside the server engine.
For example, Spring Kafka's DefaultErrorHandler(FixedBackOff(0L, 3L)) retries 3 times with an interval of 0. The first run plus 3 retries executes one record 4 times. The default of a DefaultErrorHandler with no configuration is FixedBackOff(0L, 9), so it executes one record 10 times.
| Item | Driver retryWrites | Application retry (Spring Kafka example) |
|---|---|---|
| Setting | retryWrites=true (default for 4.2-compatible drivers) |
The BackOff of DefaultErrorHandler |
| When | Network error, no primary found | When the listener throws an exception |
| How many times | 1 more (when timeoutMS is not used) |
As many retries as configured |
| What is redone | The one failed command (for a bulk, that batch) | The whole listener processing |
| Interval | As soon as a server is selected again | The wait set by BackOff |
| As the server sees it | Same lsid and txnNumber. Same write |
New lsid or new txnNumber. New write |
Below is a hypothetical example. The numbers are also for illustration.
store_inventory collection.socketTimeoutMS is 30 seconds, and retryWrites is at its default (on).DefaultErrorHandler(FixedBackOff(0L, 3L)) and sends the record to a DLQ (dead letter topic) when everything fails.| Time | On the client | On the server (assumed) |
|---|---|---|
| 0s | 1st application execution. Bulk sent (session A, number 1) | Copy 1 starts running |
| 30s | Socket timeout. The driver resends the same bulk (A, 1) | The resend waits for session A. 1 copy |
| 60s | The resend also times out. The exception reaches the application. 2nd application execution (B, 1) | Copy 2 starts running. Copy 1 keeps running |
| 90s | Driver resend (B, 1) | Waiting for session B. 2 copies |
| 120s | 3rd application execution (C, 1) | 3 copies |
| 180s | 4th application execution (D, 1) | 4 copies |
| 240s | The resend of the 4th execution also times out. Retries exhausted. Sent to DLQ | 4 copies still running |
At 240 seconds, when the application gives up on the record, the server still has 4 copies of the same bulk and 4 resends waiting for their sessions. Once the server operation unblocks, all 4 copies run.
서버로 간 전송 최대 = 애플리케이션 실행 수 × 2
엔진에서 겹쳐 도는 사본 최대 = 애플리케이션 실행 수
In the example above, the number of application executions is 4. There are at most 8 sends and at most 4 copies. With the default FixedBackOff(0L, 9), it is 20 sends and 10 copies.
In two cases, fewer retries happen and the maximum is not reached.
This calculation is for one record. If several consumers fall into the same situation, the number of copies is multiplied again by the number of consumers.
If you set maxTimeMS shorter than the socket timeout, the server stops the operation first, before the client gives up, and returns MaxTimeMSExpired. This error is not a network error, so the driver does not resend. The retry overlap shrinks by that much.
There are three grounds for MaxTimeMSExpired (code 50) not being retryable.
ExceededTimeLimit (262) in the table is a different code.RetryableWriteError label only to errors in the RetriableError category (src/mongo/db/error_labels.cpp, r4.4.0). In error_codes.yml at the same tag, the categories of MaxTimeMSExpired are Interruption and ExceededTimeLimitError.RETRYABLE_ERROR_CODES list of Java driver r5.4.0.maxTimeMS also clears out queued operations. MongoDB limits the number of writes entering the engine at once with tickets (admission passes). Waits for a ticket or for the global lock are interruptible, so operations still waiting drop out once the deadline passes.
An operation inside the engine that cannot reach an interrupt check point cannot be cut off by maxTimeMS either. The WiredTiger loop in part 3/6 is an example. In that case the server cannot keep the deadline, so the socket timeout fires first and the multiplication above happens as is.
In Spring Data MongoDB, you can set maxTimeMS with Query.maxTime(Duration) only for queries. BulkOperations has no option to pass it. maxTime does not appear even once in the BulkOperations and DefaultBulkOperations source of Spring Data MongoDB 4.4.0.
The Java driver's CSOT (Client-Side Operation Timeout, timeoutMS) is a time limit that also covers server execution. The official documentation says that when timeoutMS is set, a write can be retried multiple times. In that case the doubling calculation above does not hold.
currentOp, you see more writes against the same collection than the number of application executions. If you group them by lsid and txnNumber, the number of groups is the number of copies.FixedBackOff with interval 0 stacks copies up immediately. Use ExponentialBackOffWithMaxRetries to widen the interval, and add jitter (random variation) if there are many consumers. Even with an interval, copies pile up while the server is stalled. The interval slows the rate at which they pile up and buys time for the server to recover.maxTimeMS shorter than the socket timeout. For example, set maxTimeMS to 20 seconds and socketTimeoutMS to 30 seconds. If the server stops first, no driver resend happens.max.poll.interval.ms (Kafka default 300000ms). If it goes over, the consumer leaves the group and the offset commit fails (Korean). Spring Kafka's default BackOffHandler puts the consumer thread to sleep while it waits. A generous reference formula is below.max.poll.records × (한 번 실행 최대 시간 × 실행 수 + 대기 합) < max.poll.interval.ms
_id gives the same result no matter how many times it overlaps. An insert whose _id the driver generates fresh each time adds one more document per application retry.updateMany or deleteMany. On failure you cannot tell whether only part of it was applied, so you have to judge separately whether an application retry is safe.DefaultErrorHandler.addNotRetryableExceptions(...).
- Do not read a socket timeout as a cancellation. A write the client gave up on keeps running on the server.
- Turning off retryWrites does not remove the overlap. The number of server copies is set by the number of application executions. Turning off retryWrites only halves the sends to the server.
- retryWrites does not protect
deleteMany. On a network error, the driver throws an exception without resending, and you cannot tell how many documents were deleted.- Do not use
maxTimeMSas a universal deadline. An operation inside the engine that cannot reach an interrupt check point keeps running past the deadline.
This procedure was not verified by running it. The results below are expectations from reading the source and the specifications.
You need two things.
--setParameter enableTestCommands=1. In 4.2.12, configureFailPoint is registered as a test command (MONGO_REGISTER_TEST_COMMAND in src/mongo/db/commands/fail_point_cmd.cpp).The procedure is as follows.
failCommand fail point. It holds each insert from connections whose appName is retry-lab for 60 seconds.db.adminCommand({
configureFailPoint: "failCommand",
mode: { times: 8 },
data: { failCommands: ["insert"], blockConnection: true,
blockTimeMS: 60000, appName: "retry-lab" }
})
socketTimeoutMS=5000&retryWrites=true&appName=retry-lab._id.for (let i = 1; i <= 4; i++) {
try { db.retry_lab.insertOne({ productId: 1, storeId: 7, attempt: i }); break; }
catch (e) { print(`attempt ${i}: ${e.name}`); }
}
db.getSiblingDB("admin").aggregate([
{ $currentOp: { allUsers: true } },
{ $match: { "command.insert": "retry_lab" } },
{ $group: { _id: { lsid: "$command.lsid.id", txn: "$command.txnNumber" },
n: { $sum: 1 } } }
])
db.retry_lab.countDocuments({ productId: 1 }).If the shell's driver resends after a socket timeout, the expected results are as follows.
attempt 1 through attempt 4 are printed one at a time, about every 10 seconds. One execution spends 5 seconds on the original send and 5 seconds on the resend.n of 2. The fail point holds the command at service_entry_point_common.cpp:715. Session checkout happens later, inside the runCommandImpl call at line 912. So in this experiment, the resend does not wait for the session either; it is held at the fail point along with the original request._id values. The 4 driver resends do not add documents.
- Server source for the session wait: The statement that the resend waits until the original request finishes is based on the driver sessions specification. It was not checked directly against
session_catalog.cppin the 4.2 server.- Sharded cluster path: The path by which mongos passes
lsidandtxnNumberto the shard and the shard's mongod checks out the session was not traced in the source. The server behavior described on this page is based on mongod.- Scope of CSOT: The version that introduced Java driver
timeoutMS, how far it applies to bulk writes, and how many retries happen in that case were not checked.- Jitter support: Whether Spring's
ExponentialBackOfffamily supports jitter was not checked per version.- The driver in the reproduction procedure: Whether the Node.js driver used by mongosh resends a write after a socket timeout was not checked in that driver's source. If it does not resend,
nin step 4 is 1.- Field location in
$currentOp: WhetherlsidandtxnNumberappear as is inside thecommanddocument was not confirmed by running it.- Latest server versions: Whether the list of commands that receive the kill mark on disconnect changed after 4.2 was not checked. 4.2 is a version whose support has ended.