Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Python Condition Variable: Synchronizing Threads with wait and notify

A Python condition variable coordinates threads with wait() and notify(). Learn about predicate loops, producer-consumer code, handling spurious wakeups and timeouts, and when queue.Queue is simpler.

~4 min readPython
#threading#concurrency#synchronization#condition variable
Illustration of two threads coordinating through a Python condition variable, with a wait and notify metaphor

A Python condition variable lets one or more threads wait until another thread signals a state change. It is part of the threading module and is commonly used for producer-consumer patterns where a worker thread must wait for data to become available. The core idea is simple: a thread acquires an associated lock, checks a predicate, and if the predicate is false, it calls wait() to release the lock and sleep. When another thread changes the state, it calls notify() or notify_all() to wake the waiting thread, which then reacquires the lock and rechecks the predicate.

What a Condition Variable Does in Python

A threading.Condition object is always tied to a lock. You can create it with an existing lock or let it create its own RLock by default. The condition provides three essential methods:

  • wait(timeout=None) — releases the lock, blocks until notified or timeout expires, then reacquires the lock.
  • notify(n=1) — wakes up at most n waiting threads.
  • notify_all() — wakes up all waiting threads.

All three must be called while the thread holds the associated lock. The lock is what protects the shared state you are waiting on. Without it, you risk race conditions where a notification arrives between checking a predicate and going to sleep.

Minimal Example: Waiting for a State Change

Here is a minimal scenario: one thread waits for a flag to become true, and another thread sets it.

python
import threading import time condition = threading.Condition() flag = False def waiter(): global flag with condition: while not flag: print("Waiter: waiting") condition.wait() print("Waiter: flag is now True") def setter(): global flag time.sleep(1) with condition: flag = True print("Setter: setting flag and notifying") condition.notify() t1 = threading.Thread(target=waiter) t2 = threading.Thread(target=setter) t1.start() t2.start() t1.join() t2.join()

The while not flag loop is not optional. It guards against spurious wakeups and against the possibility that another thread consumed the notification before this thread reacquired the lock. Always recheck the predicate after wait() returns.

Producer-Consumer with a Condition Variable

The classic use case is a shared buffer where a producer adds items and a consumer removes them. The consumer must wait when the buffer is empty, and the producer must notify when it adds an item.

python
import threading import time import random class Buffer: def __init__(self, capacity): self.capacity = capacity self.items = [] self.condition = threading.Condition() def put(self, item): with self.condition: while len(self.items) >= self.capacity: self.condition.wait() self.items.append(item) self.condition.notify() def get(self): with self.condition: while not self.items: self.condition.wait() item = self.items.pop(0) self.condition.notify() return item def producer(buffer, count): for i in range(count): buffer.put(i) print(f"Produced {i}") time.sleep(random.uniform(0.1, 0.5)) def consumer(buffer, count): for _ in range(count): item = buffer.get() print(f"Consumed {item}") time.sleep(random.uniform(0.2, 0.6)) buffer = Buffer(5) p = threading.Thread(target=producer, args=(buffer, 10)) c = threading.Thread(target=consumer, args=(buffer, 10)) p.start() c.start() p.join() c.join()

Notice that both put and get use notify() after modifying the buffer. This wakes up a single waiting thread. If multiple consumers are waiting, notify() wakes only one, which is usually sufficient. If you have multiple producers and consumers, you may need notify_all() to avoid starvation.

Handling Spurious Wakeups and Timeouts

Condition.wait() can return before the predicate is true. This can happen because another thread changed the state before this thread reacquired the lock, and implementations may also permit spurious wakeups. The while loop around the predicate is therefore the standard defense.

You can also pass a timeout to wait(). This is useful when you want to give up after a certain period, for example in a polling loop or when handling shutdown signals.

python
def wait_with_timeout(condition, predicate, timeout): with condition: while not predicate(): if not condition.wait(timeout): return False return True

If the timeout expires, wait() returns False; if it was notified, it returns True. The predicate must still be checked, because a notification does not guarantee the predicate is now true.

Condition Variable vs Event and Queue

Python provides other synchronization primitives that may be simpler for specific use cases. A threading.Event is a boolean flag that threads can wait on. It is simpler but does not support complex predicates or multiple state changes. A queue.Queue internally uses condition variables and locks, and is often the best choice for producer-consumer communication because it handles buffering, blocking, and thread safety for you.

PrimitiveBest forCondition variable needed?
threading.EventSignaling a simple boolean flagNo, but limited to boolean state
queue.QueueProducer-consumer with a bufferNo, already implemented
threading.ConditionCustom predicates or multi-step synchronizationYes

Use a condition variable when you need to wait for a specific condition that is more complex than a simple flag, or when you need fine-grained control over notification behavior. For straightforward FIFO data exchange, queue.Queue is less error-prone.

Deadlock and Performance Considerations

Condition variables can deadlock if a notification is lost or if a thread never rechecks its predicate, so always use a predicate while loop and call methods with with condition:. Calling notify() or notify_all() without holding the lock also raises RuntimeError.

Another common mistake is calling notify() while holding the lock and then performing a slow operation before releasing it. The notified thread will wake up and immediately try to reacquire the lock, so it will block until you release it. This is not a deadlock, but it can cause unnecessary contention. If you have many waiting threads, notify_all() is more expensive than notify() because it wakes every thread, but it may be necessary to prevent starvation when multiple threads wait on different predicates.

From a performance perspective, condition variables are lightweight compared to busy-waiting, but they still involve OS-level thread blocking and wakeups. For a high-throughput producer-consumer workload, queue.Queue with a bounded size is usually simpler; it handles the same locking internally and avoids custom condition logic.

Advanced Usage: Multiple Conditions and Predicates

You can use a single condition variable with multiple predicates, but you must be careful with notify_all() to ensure the correct thread wakes up. Alternatively, you can use separate condition variables for different state changes. They can share the same lock, but each has its own waiter list, which can reduce unnecessary wakeups:

python
import threading lock = threading.Lock() data_ready = threading.Condition(lock) data_consumed = threading.Condition(lock)

Using separate conditions can improve responsiveness when threads wait on unrelated conditions. However, it also increases complexity because you must keep lock ordering consistent to avoid deadlocks. In practice, a single condition with a well-defined predicate is often sufficient and easier to reason about.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.