Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Python Thread Local Storage: Using threading.local

Learn how to use threading.local in Python to store per-thread data safely, avoid shared-state bugs, and understand when to prefer contextvars.

~7 min readPython
#threading#concurrency#thread-local#contextvars
Illustration of multiple threads each holding their own isolated storage compartment, representing Python thread-local data.

When multiple threads in a Python program need to keep their own copy of a value, sharing a single global variable is a common source of subtle bugs. The threading.local class provides a clean way to store data that is private to each thread. This article explains how thread-local storage works in Python, where it fits in a concurrent program, and what to watch out for when using it in production.

The Problem Thread-Local Storage Solves

In a multithreaded application, global variables are shared across all threads. If one thread modifies a global, every other thread sees that change. That is sometimes intended, but often you need per-thread state: a database connection, a request context, or a user identity that should not leak between threads. Without thread-local storage, you would have to pass that state through every function call or protect it with locks, which quickly becomes unwieldy.

Thread-local storage gives each thread its own independent binding for an attribute. In Python, threading.local is the standard way to achieve this. Each thread that accesses a threading.local object sees a separate set of attributes, so writes in one thread do not affect reads in another.

Using threading.local in Python

The threading.local class is part of the standard library. You create an instance and then assign attributes on it. Here is a minimal example:

python
import threading local = threading.local() def worker(): local.value = 42 print(f"Thread {threading.current_thread().name} sees {local.value}") threads = [threading.Thread(target=worker) for _ in range(3)] for t in threads: t.start() for t in threads: t.join()

Each thread sets local.value independently. The attribute does not exist on the main thread unless you set it there. Accessing an unset attribute raises AttributeError, which is useful for detecting logic errors.

You can also initialize thread-local data lazily. A common pattern is to check for an attribute and create it if missing:

python
def get_connection(): if not hasattr(local, "conn"): local.conn = create_connection() return local.conn

This avoids passing connection objects through every function and keeps the connection tied to the thread that created it.

How Thread-Local Data Behaves Across Threads

Thread-local storage is not magic. The interpreter keeps a separate namespace for each thread that touches a threading.local object. When a thread first uses the object, Python creates a separate namespace for that thread. The important consequence is that one thread cannot see another thread's attribute bindings. This isolation is the core guarantee. Keep in mind that isolation applies to attribute bindings, not to the objects they point to: if you store the same mutable object in every thread's namespace, those threads still share that object and can mutate it concurrently.

Consider a scenario where one thread sets a value and another thread tries to read it:

python
import threading local = threading.local() local.name = "main" def worker(): print(local.name) # AttributeError: no attribute 'name' threading.Thread(target=worker).start()

The worker thread raises AttributeError because it has its own empty namespace. This is often desirable, but it also means you cannot rely on thread-local data to communicate between threads. If you need to share a value, use a regular global with appropriate synchronization.

Common Use Cases for Thread-Local Data

Thread-local storage is most valuable when you have resources that are not safe to share across threads or when you need to carry context without threading it through every call.

Database Connections

Database connections are often not thread-safe by default. A connection created in one thread should not be used in another. Storing the connection in thread-local data ensures each thread gets its own connection, and you avoid creating a new one on every operation.

Request Context in Web Frameworks

Web frameworks often need to expose the current request without passing it through every call. Older WSGI frameworks did this with thread-local storage, and some libraries still do. Modern async-aware frameworks generally use contextvars instead, because one OS thread can host many coroutines with different logical requests. If you are writing application code, use the mechanism your framework provides; do not add your own threading.local request context to an async framework.

Logging Context

You might want to include a request ID or user ID in log messages. Storing that context in thread-local data lets a logging formatter read it without passing it to every log call. This is convenient, but remember to clear the context so it does not leak between tasks in a pooled thread.

Thread-Local vs. Other Concurrency Primitives

Thread-local storage is not the only way to manage per-thread state. It is important to understand when it is the right tool and when alternatives are better.

ApproachSharing modelTypical use case
threading.localPer-thread, isolatedResources that are not thread-safe
Global variableShared across all threadsConfiguration that never changes
Lock-protected globalShared, synchronizedMutable state that must be consistent
contextvarsPer-context, propagates to async tasksRequest context in asyncio

A global variable with a lock is appropriate when you genuinely need shared state and are willing to serialize access. threading.local is for data that should never be shared. contextvars is a more modern alternative, especially for asynchronous code, because it propagates context across await points, which thread-local storage does not do.

Performance and Memory Considerations

Thread-local storage has a small runtime cost. Each attribute access on a threading.local object involves an extra lookup to find the current thread's namespace. This is slightly slower than accessing a regular attribute on a plain object, but the overhead is usually negligible compared to the cost of creating a new resource like a database connection.

Memory is a more significant concern. Each thread that touches a threading.local object keeps its own namespace for as long as the thread runs. If you create many threads or store large objects, memory usage can grow. More importantly, if you use thread pools where threads are reused, thread-local data persists across tasks unless you explicitly clear it. This can cause stale data to leak from one request to the next.

To avoid leaks, clear thread-local attributes at the end of a task. A common pattern is:

python
def handle_request(): try: # set up context local.user = get_user() # do work finally: del local.user

Using del removes the attribute and frees the reference, preventing memory bloat in long-lived threads.

Pitfalls and Edge Cases

Thread-local storage is simple, but there are several ways to misuse it.

Thread Pools and Reused Threads

In a thread pool, the same thread executes many tasks. If you set thread-local data in one task and do not clear it, the next task on that thread will see the previous task's data. This is a classic source of cross-request contamination. Always clear or reset thread-local state at the beginning or end of each task.

Inheritance and Child Threads

Thread-local data is not inherited by child threads. When you start a new thread, it starts with an empty namespace. If you need to pass context to a child thread, pass it explicitly as a constructor argument.

Module-Level Instances

A threading.local instance is process-wide, but it still provides per-thread namespaces. If you import the same instance in multiple modules, all of those modules refer to the same per-thread attributes. That is often fine, but use separate instances for unrelated state so different parts of the program do not accidentally read attributes set by each other.

Attribute Deletion

Deleting an attribute with del local.x removes it only for the current thread. Other threads still have their own bindings. This is consistent with the isolation model but can be surprising if you expect a global deletion.

When to Prefer contextvars Over threading.local

contextvars is a more recent addition to Python's standard library, introduced in Python 3.7. It solves a similar problem and is especially useful for asynchronous programming. Context variables are copied when a new task is created, so they naturally propagate across await calls. Thread-local storage does not work well with asyncio because a single thread can run many coroutines, and thread-local data would be shared among them.

If you are writing synchronous, thread-based code, threading.local is straightforward and efficient. If you are writing asynchronous code or need context to flow through await points, use contextvars. Choose the tool that matches your concurrency model.

Final Implementation Guidance

When you decide to use threading.local, follow a few practical rules. Create one threading.local instance per concern, not one giant object with many unrelated attributes. Use descriptive attribute names. Always clear thread-local data when a unit of work finishes, especially in long-lived threads. Do not rely on thread-local data to pass information between threads; it is strictly per-thread. If you find yourself fighting the isolation, reconsider whether thread-local storage is the right abstraction.

Thread-local storage is a small but powerful feature. Used correctly, it keeps concurrent code clean and avoids the need to thread context through every function call. Used carelessly, it introduces hidden state that is hard to debug. Understanding its behavior and limitations lets you apply it where it fits and avoid it where it does not.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.