Python Condition Variable: Synchronizing Threads with wait and notify
A Python condition variable coordinates threads with wait() and notify(). Learn about predicate loops, producer-consumer code, handling spurious wakeups and timeouts, and when queue.Queue is simpler.

A Python condition variable lets one or more threads wait until another thread signals a state change. It is part of the threading module and is commonly used for producer-consumer patterns where a worker thread must wait for data to become available. The core idea is simple: a thread acquires an associated lock, checks a predicate, and if the predicate is false, it calls wait() to release the lock and sleep. When another thread changes the state, it calls notify() or notify_all() to wake the waiting thread, which then reacquires the lock and rechecks the predicate.
What a Condition Variable Does in Python
A threading.Condition object is always tied to a lock. You can create it with an existing lock or let it create its own RLock by default. The condition provides three essential methods:
wait(timeout=None)— releases the lock, blocks until notified or timeout expires, then reacquires the lock.notify(n=1)— wakes up at mostnwaiting threads.notify_all()— wakes up all waiting threads.
All three must be called while the thread holds the associated lock. The lock is what protects the shared state you are waiting on. Without it, you risk race conditions where a notification arrives between checking a predicate and going to sleep.
Minimal Example: Waiting for a State Change
Here is a minimal scenario: one thread waits for a flag to become true, and another thread sets it.
import threading
import time
condition = threading.Condition()
flag = False
def waiter():
global flag
with condition:
while not flag:
print("Waiter: waiting")
condition.wait()
print("Waiter: flag is now True")
def setter():
global flag
time.sleep(1)
with condition:
flag = True
print("Setter: setting flag and notifying")
condition.notify()
t1 = threading.Thread(target=waiter)
t2 = threading.Thread(target=setter)
t1.start()
t2.start()
t1.join()
t2.join()The while not flag loop is not optional. It guards against spurious wakeups and against the possibility that another thread consumed the notification before this thread reacquired the lock. Always recheck the predicate after wait() returns.
Producer-Consumer with a Condition Variable
The classic use case is a shared buffer where a producer adds items and a consumer removes them. The consumer must wait when the buffer is empty, and the producer must notify when it adds an item.
import threading
import time
import random
class Buffer:
def __init__(self, capacity):
self.capacity = capacity
self.items = []
self.condition = threading.Condition()
def put(self, item):
with self.condition:
while len(self.items) >= self.capacity:
self.condition.wait()
self.items.append(item)
self.condition.notify()
def get(self):
with self.condition:
while not self.items:
self.condition.wait()
item = self.items.pop(0)
self.condition.notify()
return item
def producer(buffer, count):
for i in range(count):
buffer.put(i)
print(f"Produced {i}")
time.sleep(random.uniform(0.1, 0.5))
def consumer(buffer, count):
for _ in range(count):
item = buffer.get()
print(f"Consumed {item}")
time.sleep(random.uniform(0.2, 0.6))
buffer = Buffer(5)
p = threading.Thread(target=producer, args=(buffer, 10))
c = threading.Thread(target=consumer, args=(buffer, 10))
p.start()
c.start()
p.join()
c.join()Notice that both put and get use notify() after modifying the buffer. This wakes up a single waiting thread. If multiple consumers are waiting, notify() wakes only one, which is usually sufficient. If you have multiple producers and consumers, you may need notify_all() to avoid starvation.
Handling Spurious Wakeups and Timeouts
Condition.wait() can return before the predicate is true. This can happen because another thread changed the state before this thread reacquired the lock, and implementations may also permit spurious wakeups. The while loop around the predicate is therefore the standard defense.
You can also pass a timeout to wait(). This is useful when you want to give up after a certain period, for example in a polling loop or when handling shutdown signals.
def wait_with_timeout(condition, predicate, timeout):
with condition:
while not predicate():
if not condition.wait(timeout):
return False
return TrueIf the timeout expires, wait() returns False; if it was notified, it returns True. The predicate must still be checked, because a notification does not guarantee the predicate is now true.
Condition Variable vs Event and Queue
Python provides other synchronization primitives that may be simpler for specific use cases. A threading.Event is a boolean flag that threads can wait on. It is simpler but does not support complex predicates or multiple state changes. A queue.Queue internally uses condition variables and locks, and is often the best choice for producer-consumer communication because it handles buffering, blocking, and thread safety for you.
| Primitive | Best for | Condition variable needed? |
|---|---|---|
threading.Event | Signaling a simple boolean flag | No, but limited to boolean state |
queue.Queue | Producer-consumer with a buffer | No, already implemented |
threading.Condition | Custom predicates or multi-step synchronization | Yes |
Use a condition variable when you need to wait for a specific condition that is more complex than a simple flag, or when you need fine-grained control over notification behavior. For straightforward FIFO data exchange, queue.Queue is less error-prone.
Deadlock and Performance Considerations
Condition variables can deadlock if a notification is lost or if a thread never rechecks its predicate, so always use a predicate while loop and call methods with with condition:. Calling notify() or notify_all() without holding the lock also raises RuntimeError.
Another common mistake is calling notify() while holding the lock and then performing a slow operation before releasing it. The notified thread will wake up and immediately try to reacquire the lock, so it will block until you release it. This is not a deadlock, but it can cause unnecessary contention. If you have many waiting threads, notify_all() is more expensive than notify() because it wakes every thread, but it may be necessary to prevent starvation when multiple threads wait on different predicates.
From a performance perspective, condition variables are lightweight compared to busy-waiting, but they still involve OS-level thread blocking and wakeups. For a high-throughput producer-consumer workload, queue.Queue with a bounded size is usually simpler; it handles the same locking internally and avoids custom condition logic.
Advanced Usage: Multiple Conditions and Predicates
You can use a single condition variable with multiple predicates, but you must be careful with notify_all() to ensure the correct thread wakes up. Alternatively, you can use separate condition variables for different state changes. They can share the same lock, but each has its own waiter list, which can reduce unnecessary wakeups:
import threading
lock = threading.Lock()
data_ready = threading.Condition(lock)
data_consumed = threading.Condition(lock)Using separate conditions can improve responsiveness when threads wait on unrelated conditions. However, it also increases complexity because you must keep lock ordering consistent to avoid deadlocks. In practice, a single condition with a well-defined predicate is often sufficient and easier to reason about.
Try matching browser tools
All toolsSmall utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Python guides
All Python guidesPython iter(): How the Built-in Function Works
Explains Python's iter() built-in: how it creates iterators, the two-argument sentinel form, custom iterator classes, and common edge cases.
Python Pika: Queues, Exchanges, Routing Keys, and Acknowledgements
Learn how to use Python's Pika client with RabbitMQ: declare queues and exchanges, publish with routing keys, consume messages, and handle acknowledgements reliably.
Python Kafka: kafka-python vs Confluent Client
python kafka python vs confluent kafka: Compare kafka-python and Confluent Kafka Python clients: API differences, performance, feature coverage, and when to choose eac...
Python Kafka Offsets, Partitions, and Manual Commits
Learn how Kafka offsets and partitions work and how to manually commit offsets in Python consumers with kafka-python and confluent-kafka. Covers sync vs async commits, rebalances, and seeking.
More in Python
All Python guidesThe guides filed next to this one in Python, in library order.