PyMongo Transactions: Sessions, Commit/Abort, and Retries

PyMongo transactions require a MongoDB deployment that supports multi-document transactions. In practice, that means a replica set or a sharded cluster whose shards are replica sets. A standalone mongod instance does not support transactions, so confirm your server topology before writing transaction code.

What MongoDB Transactions Require

MongoDB transactions are available starting in version 4.0 for replica sets and 4.2 for sharded clusters. PyMongo exposes transaction support through the session API. The key requirement is that every operation you want to include in a transaction must be executed within the same session.

You also need to consider the storage engine. The WiredTiger storage engine, which is the default since MongoDB 3.2, supports transactions. Older engines such as MMAPv1 do not support transactions; MMAPv1 was removed in MongoDB 4.2, so upgrade or migrate before using transactions.

Transactions also have a server-side lifetime limit. The default is 60 seconds, controlled by transactionLifetimeLimitSeconds. The server aborts a transaction that exceeds the limit, so design transaction bodies to complete quickly.

Starting a Session and Transaction

In PyMongo, you create a client, then start a session using client.start_session(). Inside that session, you call session.start_transaction(). All operations that participate in the transaction must be executed with the session argument passed to the collection methods.

from pymongo import MongoClient

client = MongoClient("mongodb://localhost:27017")

with client.start_session() as session:
    with session.start_transaction():
        db = client.test_db
        db.accounts.update_one(
            {"user": "alice"},
            {"$inc": {"balance": -100}},
            session=session,
        )
        db.accounts.update_one(
            {"user": "bob"},
            {"$inc": {"balance": 100}},
            session=session,
        )

The with blocks ensure that the session is closed and the transaction is committed on normal exit or aborted if an exception occurs. If both updates succeed, the transaction is committed when the with block exits normally. If any operation raises an exception, the transaction is aborted automatically.

Committing and Aborting a Transaction

You can also manage the transaction explicitly instead of relying on the context manager. The start_transaction() method returns a transaction object, and you can call commit_transaction() or abort_transaction() on the session.

session = client.start_session()
session.start_transaction()
try:
    db.accounts.update_one(
        {"user": "alice"},
        {"$inc": {"balance": -100}},
        session=session,
    )
    db.accounts.update_one(
        {"user": "bob"},
        {"$inc": {"balance": 100}},
        session=session,
    )
    session.commit_transaction()
except Exception:
    session.abort_transaction()
finally:
    session.end_session()

When you commit, MongoDB writes all changes atomically. If you abort, none of the operations are applied. If commit_transaction() cannot confirm the result because of a network failure or a primary change, the commit may or may not have succeeded; retry the commit when the error has the UnknownTransactionCommitResult label, and retry the whole transaction only when the error has the TransientTransactionError label.

Handling Errors and Retry Logic

Transactions can fail for several reasons: transient network issues, write conflicts, or timeouts. PyMongo exposes error labels on OperationFailure so you can choose a safe retry strategy. The recommended pattern is to catch OperationFailure and check for a TransientTransactionError label.

from pymongo.errors import OperationFailure

def run_transaction_with_retry(session, fn):
    while True:
        try:
            with session.start_transaction():
                fn(session)
            return
        except OperationFailure as exc:
            if exc.has_error_label("TransientTransactionError"):
                continue
            raise

This loop retries the entire transaction when the server indicates a transient error. For commit errors with the UnknownTransactionCommitResult label, you should retry the commit operation itself, not the whole transaction.

def commit_with_retry(session):
    while True:
        try:
            session.commit_transaction()
            return
        except OperationFailure as exc:
            if exc.has_error_label("UnknownTransactionCommitResult"):
                continue
            raise

Transaction Options: Read and Write Concern

You can specify read concern, write concern, and read preference for a transaction. These are set when calling start_transaction(). The default read concern is "snapshot", which gives you a consistent view of the data. Write concern defaults to the cluster's setting.

from pymongo import ReadPreference
from pymongo.write_concern import WriteConcern

with client.start_session() as session:
    with session.start_transaction(
        read_concern={"level": "majority"},
        write_concern=WriteConcern(w="majority", j=True),
        read_preference=ReadPreference.PRIMARY,
    ):
        # operations here

Using majority write concern ensures that the transaction is durable only after the data is replicated to a majority of nodes. This increases safety but adds latency. The read preference must be PRIMARY for transactions; secondary reads are not allowed inside a transaction.

Performance and Operational Considerations

Transactions have more overhead than single-document operations. They require coordination between the server and the client and hold locks on the documents involved. To minimize contention, keep transactions short and avoid network calls or external I/O inside the transaction.

Another operational concern is transactionLifetimeLimitSeconds. If a transaction exceeds this limit, the server aborts it. You can adjust the value, but the change affects all clients. Monitoring the currentOp command can help you identify long-running transactions.

Finally, transactions have DDL restrictions. Explicitly creating or dropping collections and creating indexes are not generally allowed inside a transaction. Most transaction work is CRUD on existing collections; if you rely on implicit collection creation, check the behavior for your MongoDB version.

Common Pitfalls and Limitations

One common mistake is forgetting to pass the session argument to every operation. If an operation does not use the session, it executes outside the transaction, breaking atomicity. Always verify that every insert, update, delete, or find that should be part of the transaction includes session=session.

Another pitfall is using a transaction on a standalone server. The first transaction operation will fail with an OperationFailure such as "Transaction numbers are only allowed on a replica set member or mongos". Always confirm your deployment topology before writing transaction code.

Finally, changing the transaction read concern away from snapshot changes isolation guarantees. For example, read_concern={"level": "local"} does not provide snapshot isolation, so concurrent transactions can observe different states. Use it only when you understand the consistency implications.

Use transactions when you need to move data between collections or perform several updates atomically. The key is to use sessions correctly, handle retryable errors with the right retry strategy, and keep operations short to avoid contention and server-side transaction expiry.