Python async with vs with: Choosing the Right Context Manager

In asynchronous Python, the choice between with and async with depends on whether a context manager's setup and teardown need to perform awaitable operations. This article explains the difference and when to use each in asyncio-based applications.

What with Does in Python

The with statement invokes the __enter__ and __exit__ methods of a context manager. It calls them synchronously and does not await anything they return. When you write:

with open('file.txt') as f:
    data = f.read()

the expression is evaluated, then __enter__ is called before the block and __exit__ after it. If a context manager performs a blocking operation, such as performing file I/O or acquiring a lock, that operation blocks the calling thread. In an asyncio application, blocking the thread running the event loop prevents other coroutines from running.

What async with Adds

The async with statement is designed for asynchronous context managers. It calls __aenter__ and __aexit__ and awaits what they return; these are commonly coroutine functions. This allows setup and teardown to await I/O operations without blocking the event loop. For example:

import asyncio

class AsyncResource:
    async def __aenter__(self):
        await asyncio.sleep(1)  # Simulate async setup
        return self

    async def __aexit__(self, exc_type, exc, tb):
        await asyncio.sleep(0.5)  # Simulate async cleanup

async def main():
    async with AsyncResource() as res:
        print('Using resource')

asyncio.run(main())

Here, __aenter__ and __aexit__ are coroutines, so the event loop remains responsive while their await expressions are suspended.

When to Use with in Async Code

You can use with inside an async function as long as the context manager's __enter__ and __exit__ do not perform blocking operations. A simple context manager that sets a flag is fine:

class Flag:
    def __enter__(self):
        self.active = True
        return self

    def __exit__(self, exc_type, exc, tb):
        self.active = False


async def task():
    with Flag() as flag:
        await asyncio.sleep(0.1)

The same is not true for a threading.Lock: if the lock is contended, with lock blocks the event loop while waiting for another thread to release it. Holding a thread lock across an await is especially risky because another coroutine on the same thread can block trying to acquire it. Use an asyncio lock instead when the lock may need to be held across an await:

import asyncio

lock = asyncio.Lock()

async def task():
    async with lock:
        await asyncio.sleep(0.1)

Rule of thumb: use with when the context manager's setup and teardown are synchronous and fast. If they involve I/O, network calls, or other operations that would block, use async with.

When async with Is Required

async with is required when the context manager implements __aenter__ and __aexit__ instead of __enter__ and __exit__. Many asyncio-aware libraries provide async context managers. For example, an HTTP client session in aiohttp:

import aiohttp

async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

Trying to use with on an object that only provides __aenter__ fails because the object has no __enter__. Similarly, async with requires __aenter__ and __aexit__; a synchronous context manager without those methods cannot be used in an async with statement.

Runtime Behavior: Blocking the Event Loop

The most critical difference is how each statement affects the asyncio event loop. When you use with inside a coroutine, the __enter__ and __exit__ methods execute synchronously. If they perform blocking operations, the entire event loop stalls, defeating the purpose of async programming.

Consider a file read using the standard open() inside an async function:

async def read_file():
    with open('large_file.txt') as f:
        data = f.read()  # Blocks the event loop
    return data

While read_file is running, no other coroutine can execute. If you have multiple concurrent tasks, they all wait until the file read completes. To avoid this, use an async file library such as aiofiles:

import aiofiles

async def read_file_async():
    async with aiofiles.open('large_file.txt') as f:
        data = await f.read()
    return data

Here, aiofiles provides an async context manager that yields control to the event loop during I/O.

Error Handling Differences

The exception handling behavior is similar for both statements. If __enter__ raises, __exit__ is not called. Likewise, if __aenter__ raises, __aexit__ is not called. When __aenter__ is a coroutine, the exception propagates at the point where the coroutine is awaited, so catch exceptions around the async with block as you would with a regular with.

Both __exit__ and __aexit__ receive the exception type, value, and traceback. They can suppress an exception by returning a truthy value. The async with statement awaits __aexit__, but the suppression semantics are the same.

Practical Example: File I/O vs Async HTTP Client

To see the difference in a realistic scenario, consider a function that reads an API URL from a file and then makes an HTTP request. Using with for the file read blocks the event loop, while using async with for the HTTP client keeps it responsive:

import aiohttp

async def process():
    # Blocking file read
    with open('api_url.txt') as f:
        api_url = f.read().strip()

    # Non-blocking HTTP request
    async with aiohttp.ClientSession() as session:
        async with session.get(api_url) as resp:
            data = await resp.json()
    return data

If you have many concurrent process() calls, the file reads will serialize and block the loop. Replacing the file read with aiofiles and async with allows the loop to handle other tasks while the file is being read.

Choosing Between with and async with

The decision depends on the context manager's implementation and the nature of the operation:

Use with when the context manager is synchronous and its setup/teardown are fast or non-blocking. Use async with when the context manager provides async methods, or when you need to await during setup or teardown.

Common Mistakes and How to Avoid Them

A common mistake is using with on an async context manager; this fails because the object does not have __enter__. Another is using async with on a synchronous context manager; this fails because the object does not have __aenter__. Check the library's documentation to see which protocol it implements. If you control the context manager class, you can implement both sets of methods to support both usage patterns, but that is rare.

Another issue is mixing with and async with in the same code path without understanding the blocking implications. Even if a synchronous context manager is fast, if it does I/O, it will block. For high-concurrency applications, prefer async context managers for I/O-bound resources.

Finally, async with is only valid inside a coroutine or async generator. You cannot use it in a synchronous function. If you need to use an async context manager from synchronous code, you must run the coroutine with asyncio.run() or similar, which creates a new event loop.

When you implement your own context managers, decide whether they need to be async based on whether their setup or teardown needs to await; for example, for I/O or asyncio synchronization. If so, implement __aenter__ and __aexit__ as coroutines. If they do not, keep them synchronous to avoid unnecessary overhead. The key is to match the context manager's behavior to the execution context so the event loop stays responsive.