Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

PyMongo Unique Indexes in MongoDB: Creation and Performance

Learn how to create unique indexes with PyMongo, what they guarantee, and how they affect read and write performance.

~6 min readPython
#PyMongo#MongoDB#Indexes#Unique Indexes
A visual metaphor showing a MongoDB document collection with a unique index enforcing a single path for each key, highlighting the performance tradeoff between reads and writes.

When a MongoDB collection grows, queries without a matching index can force a collection scan. PyMongo's create_index method makes it straightforward to define indexes, but unique indexes carry specific guarantees and performance trade-offs. This article explains how to create unique indexes with PyMongo, what they enforce, and how they affect reads and writes.

Creating Indexes with PyMongo

The create_index method on a pymongo.collection.Collection instance is the primary way to define an index. A simple single-field index is created by passing the field name and a direction constant:

python
from pymongo import MongoClient, ASCENDING client = MongoClient('mongodb://localhost:27017') db = client['example_db'] collection = db['users'] collection.create_index([('email', ASCENDING)])

This creates a non-unique index on the email field. The index accelerates queries that filter on email and sorts by it. Without it, MongoDB may perform a collection scan, which becomes slower as the collection grows.

For compound indexes, pass multiple field-direction pairs:

python
collection.create_index([('last_name', ASCENDING), ('first_name', ASCENDING)])

The order of fields matters: this index supports queries that filter on last_name alone, but MongoDB usually cannot use it efficiently for first_name alone unless last_name is also constrained in the query.

Defining Unique Indexes

To enforce uniqueness on a field, set the unique parameter to True when calling create_index:

python
collection.create_index([('email', ASCENDING)], unique=True)

Once this index exists, MongoDB rejects any insert or update that would cause two documents to have the same value for email. The rejection surfaces as a pymongo.errors.DuplicateKeyError:

python
from pymongo.errors import DuplicateKeyError try: collection.insert_one({'email': 'alice@example.com', 'name': 'Alice'}) collection.insert_one({'email': 'alice@example.com', 'name': 'Alice Again'}) except DuplicateKeyError: print('Duplicate email rejected')

Unique indexes also apply to updates. If an update would create a duplicate value, MongoDB rejects that update and raises the same error. For single-document writes, the uniqueness check and write are atomic at the document level, so an application does not need to check for duplicates manually before writing.

Unique Indexes and Write Performance

Every index adds write overhead because MongoDB must maintain the index structure for each inserted or modified document. A unique index adds a uniqueness check before allowing the write. This check is a point lookup, which is fast, but it still consumes CPU and I/O.

For workloads with high insert or update rates, the uniqueness check can become a bottleneck, especially when the index does not fit entirely in RAM. The database may need to read index pages from disk to perform the lookup, adding latency. A non-unique index updates the index structure without enforcing uniqueness, so it avoids that extra lookup.

The performance difference is usually negligible for moderate write volumes, but it can become measurable when a collection has millions of documents and the index is large. If uniqueness is a hard requirement, the cost is unavoidable. If it is only a soft preference, consider using a non-unique index and handling duplicates in application logic instead.

Unique Indexes and Read Performance

For read operations, a unique index is often more efficient than a non-unique index when querying for a single value. Because the index guarantees that a key appears at most once, MongoDB can stop looking after the first match. In a non-unique index, the query engine may need to examine multiple entries to confirm that no further matches exist, depending on the query shape.

For equality queries on a unique field, MongoDB can use the index to fetch the document directly. For range queries, a unique index does not provide a special advantage over a non-unique index; both allow an ordered scan. The primary read benefit of a unique index is the implicit guarantee of a single match, which simplifies query planning and can reduce the number of index entries examined.

Partial and Sparse Unique Indexes

Sometimes you need uniqueness only for a subset of documents. MongoDB supports partial indexes, which include only documents that match a filter expression. In PyMongo, use partialFilterExpression:

python
collection.create_index( [('email', ASCENDING)], unique=True, partialFilterExpression={'email': {'$type': 'string'}} )

This index enforces uniqueness only on documents that have a string email value. Documents that omit email or store a non-string value are not indexed, so multiple such documents can exist. If you need to include a field whenever it exists, a sparse index or an $exists filter can work, but remember that both treat null as a present value for uniqueness checks. If null should count as absent, use a partial filter that excludes null values, such as $type.

A sparse index is a simpler form that indexes only documents that contain the indexed field. In PyMongo, set sparse=True:

python
collection.create_index([('email', ASCENDING)], unique=True, sparse=True)

Sparse indexes and partial indexes differ in expressiveness. A sparse index is equivalent to a simple existence check on the indexed field; partial indexes can use arbitrary query filters. For most cases, partial indexes are more flexible. A unique sparse index allows multiple documents to lack the indexed field, but it still treats a document with email: null as containing email.

Managing Indexes and Monitoring Performance

To list all indexes on a collection, use list_indexes():

python
for index in collection.list_indexes(): print(index)

Each index document includes the key pattern, options, and name. The default _id_ index is always present and cannot be dropped.

To remove an index, use drop_index with the index name or the key pattern:

python
collection.drop_index('email_1')

Index names are generated automatically from the key pattern unless you specify name in create_index. For complex indexes, giving an explicit name improves maintainability:

python
collection.create_index( [('email', ASCENDING)], unique=True, name='unique_email' )

Monitoring index usage is important for performance. The $indexStats aggregation stage returns usage counters for each index, including how many operations have used the index since the statistics were last reset. You can run it from PyMongo:

python
collection.aggregate([{'$indexStats': {}}])

This output helps identify unused indexes, which consume write overhead and storage without benefiting reads. Dropping unused indexes can improve write performance and reduce memory pressure.

Common Pitfalls and Operational Considerations

Creating a unique index on a collection that already contains duplicate values fails with an error. The index creation is aborted, and no index is created. You must first remove or merge duplicate documents before applying the unique constraint.

Index builds in MongoDB 4.2 and later use a hybrid approach, so the older background option is deprecated. In MongoDB 4.4 and later, commitQuorum lets you control how many replica set members must acknowledge an index build before it completes. For large collections, index creation can take time and affect database performance, so running it during low-traffic periods is advisable.

Another subtle issue is the interaction between unique indexes and sharded clusters. In a sharded collection, a unique index must include the shard key as a prefix. If you plan to shard a collection, design the unique index with the shard key in mind.

Finally, remember that a unique index enforces uniqueness at the database level, but it does not prevent race conditions in application logic that checks for existence before inserting. The only reliable way to guarantee uniqueness is to rely on the index itself and handle DuplicateKeyError in the application. Checking first and then inserting still allows two concurrent requests to pass the check before either insert completes.

Understanding how unique indexes affect both read and write performance in PyMongo is essential for designing efficient MongoDB schemas. By choosing the right index type, using partial or sparse options when appropriate, and monitoring index usage, you can maintain both data integrity and application responsiveness.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.