How to Create and Manage Threads in Python
Learn how to create and coordinate threads in Python with threading.Thread: passing arguments, daemon threads, locks, exception handling, and using ThreadPoolExecutor when you need a bounded pool of workers.

When you need to run multiple tasks concurrently in Python, the threading module provides the most direct way to create and manage threading.Thread instances. The core class is threading.Thread, which wraps a callable and executes it in a separate thread of control. This article focuses on the mechanics of creating, starting, and coordinating threads, with attention to the runtime behavior that affects real applications.
The Minimal Thread Creation Pattern
The simplest way to create a thread is to pass a target function to the Thread constructor and call start(). The thread begins executing as soon as the interpreter schedules it, which is not guaranteed to be immediate.
import threading
import time
def worker():
print("Worker starting")
time.sleep(1)
print("Worker done")
thread = threading.Thread(target=worker)
thread.start()
thread.join()The join() call blocks the main thread until the worker finishes. Without it, the main thread may continue running or reach the end of the program. At interpreter shutdown, Python waits for non-daemon threads to finish, so the worker will normally complete anyway. Calling join() gives you an explicit synchronization point when the main thread needs to wait for the worker's result.
Passing Arguments to the Target Function
Thread functions often need input data. The Thread constructor accepts args and kwargs that are forwarded to the target.
import threading
def process_item(item, retries=3):
print(f"Processing {item} with {retries} retries")
thread = threading.Thread(target=process_item, args=("order-42",), kwargs={"retries": 5})
thread.start()
thread.join()Arguments are passed positionally and by keyword exactly as they would be to a normal function call. This keeps the thread creation code readable and avoids relying on global state.
Daemon Threads and Program Exit Behavior
A daemon thread is one that does not block the interpreter from exiting when only daemon threads remain. Set daemon=True in the constructor or assign the attribute before start().
import threading
import time
def background_poll():
while True:
print("Polling...")
time.sleep(2)
t = threading.Thread(target=background_poll, daemon=True)
t.start()
print("Main continues")
# Program exits immediately; daemon thread is killed.Daemon threads are useful for background tasks that should not prevent shutdown, such as telemetry or cache warming. However, they are abruptly stopped when the interpreter exits, so they are not appropriate for work that must complete cleanly.
Thread Safety and the Global Interpreter Lock
In CPython, the GIL (Global Interpreter Lock) ensures that only one thread executes Python bytecode at a time. This makes CPU-bound Python code effectively serialized, but threads still help with I/O-bound tasks where the lock is released during blocking operations. Creating many threads for CPU-heavy work may not yield the expected speedup; multiprocessing is often a better fit for CPU-bound work, while asyncio is another option for high-concurrency I/O-bound tasks.
When threads share mutable data, race conditions can occur. The standard remedy is a threading.Lock to protect critical sections.
import threading
counter = 0
lock = threading.Lock()
def increment():
global counter
for _ in range(100000):
with lock:
counter += 1
threads = [threading.Thread(target=increment) for _ in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
print(counter)Without the lock, the increments can interfere because the read-modify-write sequence is not atomic. The with statement acquires and releases the lock even if an exception occurs.
Handling Exceptions Inside Threads
Exceptions raised inside a thread do not propagate to the caller of start() or join(). By default, an uncaught exception is reported to sys.stderr and terminates the thread. To capture exceptions, wrap the target function and store the exception in a shared structure.
import threading
import sys
def safe_target(result_container):
try:
raise ValueError("boom")
except Exception as e:
result_container["error"] = e
result = {}
t = threading.Thread(target=safe_target, args=(result,))
t.start()
t.join()
if "error" in result:
print(f"Thread failed: {result['error']}")This pattern lets the main thread inspect the outcome after joining. For more structured concurrency, consider concurrent.futures.ThreadPoolExecutor, which returns Future objects that can hold exceptions.
Joining Threads and Timeouts
join() accepts an optional timeout. If the thread does not finish within the timeout, join() returns and the main thread continues. The thread is still running in the background.
import threading
import time
def slow_worker():
time.sleep(5)
t = threading.Thread(target=slow_worker)
t.start()
t.join(timeout=2)
print(f"Thread alive after timeout: {t.is_alive()}")This is useful for graceful shutdown sequences where you cannot block indefinitely. You can check is_alive() to decide whether to force a stop or leave the thread to finish.
Practical Thread Lifecycle Management
A thread can be started only once. Calling start() a second time raises RuntimeError. After a thread finishes, it cannot be restarted. For recurring tasks, create a new thread each time or use a worker pool.
Threads consume memory for their stacks and bookkeeping until they finish. If you need to run many short-lived tasks, join them or otherwise keep the number of live threads bounded; avoid creating thousands of threads without a pool.
For most production scenarios, ThreadPoolExecutor from concurrent.futures provides a higher-level interface with a fixed number of workers, which simplifies resource management and result collection.
from concurrent.futures import ThreadPoolExecutor
def square(x):
return x * x
with ThreadPoolExecutor(max_workers=4) as executor:
results = list(executor.map(square, range(10)))
print(results)The executor handles thread creation, reuse, and shutdown automatically. Use it when you need to run many independent tasks rather than managing individual threads manually.
When Threads Are the Right Tool
Threads in Python are most effective for I/O-bound workloads: network requests, file operations, database calls, or any operation that blocks on external resources. The GIL is released during most blocking I/O, so multiple threads can make progress concurrently. For CPU-bound computation, the GIL prevents true parallel execution within one process; multiprocessing may be a better fit. If the workload is I/O-bound and can be expressed asynchronously, asyncio is another option.
Creating a thread is cheap relative to a process, but not free. Each thread has its own stack (typically several megabytes of virtual memory). If you need to handle thousands of concurrent tasks, an event loop or a thread pool with a bounded number of workers is more scalable than creating an unbounded number of threads.
Thread safety also extends to the standard library, but not uniformly. For example, an individual list.append is atomic in CPython, but compound operations that read and then modify a shared structure are not. Avoid relying on undocumented thread-safety guarantees; always protect shared mutable state with locks or use queue.Queue for producer-consumer patterns.
The threading module gives you fine-grained control over thread creation and coordination. Start with the Thread class for simple cases, move to a pool for repetitive tasks, and consider multiprocessing when the GIL becomes the bottleneck. The key is to match the concurrency model to the nature of the work, not to the number of lines of code you want to write.
Try matching browser tools
All toolsSmall utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Python guides
All Python guidesPython iter(): How the Built-in Function Works
Explains Python's iter() built-in: how it creates iterators, the two-argument sentinel form, custom iterator classes, and common edge cases.
Python Pika: Queues, Exchanges, Routing Keys, and Acknowledgements
Learn how to use Python's Pika client with RabbitMQ: declare queues and exchanges, publish with routing keys, consume messages, and handle acknowledgements reliably.
Python Kafka: kafka-python vs Confluent Client
python kafka python vs confluent kafka: Compare kafka-python and Confluent Kafka Python clients: API differences, performance, feature coverage, and when to choose eac...
Python Kafka Offsets, Partitions, and Manual Commits
Learn how Kafka offsets and partitions work and how to manually commit offsets in Python consumers with kafka-python and confluent-kafka. Covers sync vs async commits, rebalances, and seeking.
More in Python
All Python guidesThe guides filed next to this one in Python, in library order.