Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Python Async Yield: How Async Generators Work

Async generators in Python combine `async def` with `yield`. See how `async for` drives them, how `asend()` and `aclose()` work, and when a queue is a better fit.

~4 min readPython
#asyncio#async generators#coroutines#python generators
Editorial illustration of an async generator pipeline with a yield point between two await operations, conveying pull-based streaming in Python.

When a function body contains both async def and yield, Python does not treat it as a coroutine. The function becomes an async generator function, and calling it returns an async generator object. That object cannot be awaited and cannot be iterated with a plain for loop. It is driven with async for, and it may contain await expressions between yields. Understanding what it produces is the first step to using it correctly.

What an async def with yield Actually Creates

Python has three distinct function forms that look similar but behave very differently:

Function formCalling it returnsHow you consume it
def f(): yield ...generatorfor loop or next()
async def f(): yield ...async generatorasync for loop
async def f(): ...coroutineawait

The presence of yield is what switches an async def from coroutine to async generator. This is not a stylistic choice: the two forms have incompatible protocols. A coroutine returns a single value when awaited. An async generator produces a sequence of values over time, and each value may be preceded by asynchronous work.

A minimal async generator:

python
async def countdown(n: int): while n > 0: yield n n -= 1

Calling countdown(3) returns an async generator object immediately. No code in the body runs until iteration begins.

Iterating with async for

The async for loop drives the async generator. Each iteration resumes the generator until the next yield, runs the loop body, then asks for the next value.

python
import asyncio async def ticker(): for i in range(1, 4): await asyncio.sleep(0.1) yield i async def main(): async for value in ticker(): print(value) asyncio.run(main())

Between each yield, the generator can perform await operations, so the values are produced lazily as the consumer requests them. The consumer controls the pace: the generator does not run ahead and buffer values. This pull-based behavior is the defining characteristic of an async generator.

The Async Generator Protocol

Beyond __anext__, which async for uses internally, async generators expose three async methods that mirror the generator protocol:

  • asend(value) resumes the generator and makes yield evaluate to value.
  • athrow(exc) raises exc at the suspended yield point.
  • aclose() raises GeneratorExit at the suspended yield point and releases the generator.
python
async def receiver(): received = yield "ready" yield f"got: {received}" async def main(): gen = receiver() await gen.asend(None) # start the generator result = await gen.asend("ping") print(result) # got: ping await gen.aclose()

The first asend(None) is required to advance the generator to its first yield, matching how send(None) starts a regular generator. athrow is useful when you want the generator's own exception handling to observe an error from the consumer side.

Where await and yield Can Appear

An async generator may contain await before or after yield, and the placement changes when code executes relative to the consumer. In the following examples, get_data() stands in for any awaitable operation.

python
async def fetch_first(): data = await get_data() # runs before the first value is delivered yield data async def fetch_later(): yield "immediate" # delivered before any await runs data = await get_data() yield data

In fetch_first, the consumer waits for the asynchronous operation before receiving anything. In fetch_later, the first value is delivered immediately, and the await happens only when the consumer requests the second value. Choosing between these shapes controls latency and resource usage: deferring awaits keeps the first value cheap, while awaiting up front lets you validate or transform data before exposing it.

Common Failure Modes

The most frequent errors come from treating an async generator as one of the other two forms.

Using a plain for loop raises a TypeError because an async generator is not iterable in the synchronous sense:

python
for value in ticker(): # TypeError: 'async_generator' object is not iterable print(value)

Awaiting the generator directly also fails, because the object is not a coroutine:

python
await ticker() # TypeError: object async_generator can't be used in 'await' expression

A return statement with a value inside an async generator attaches that value to StopAsyncIteration. This is rarely what you want; if you need a final result, collect values in the consumer or use a separate coroutine.

A subtler issue is failing to close the generator. If you hold an async generator and stop consuming it before it finishes, cleanup is not guaranteed to run promptly. Manual iteration with __anext__ or asend does not close the generator for you. If you exit an async for loop early, do not assume the generator is closed; call aclose() explicitly or use contextlib.aclosing (Python 3.10+).

Async Generators vs. asyncio.Queue

Both async generators and asyncio.Queue can stream values between asynchronous producers and consumers, but they have opposite control flow.

An async generator is pull-based: the consumer requests each value, and the producer code runs only as needed. This is ideal when the producer is cheap to run incrementally, such as reading from a file or an HTTP stream, and when you want backpressure to propagate naturally to the producer.

An asyncio.Queue is push-based: a producer task puts values into the queue, and a consumer task gets them. This decouples the two sides and allows the producer to run ahead, buffering values. Use a queue when the producer and consumer are independent tasks, when you need bounded buffering, or when multiple consumers must share the same stream.

For a one-to-one streaming pipeline where the consumer controls the pace, an async generator is simpler and avoids the overhead of queue management.

Resource Cleanup and aclose()

Because an async generator can hold files, connections, or locks across awaits, cleanup matters. The generator's finally block runs when the generator finishes normally, when aclose() is called, or when an exception propagates through the generator. If an async for loop is stopped early, the generator may still be suspended, so close it explicitly if you need deterministic cleanup.

python
async def guarded_stream(lock: asyncio.Lock): await lock.acquire() try: for i in range(10): yield i finally: lock.release()

If you abandon an async generator without closing it, cleanup depends on the event loop's finalization behavior and may be delayed. In long-running services, this leaks resources silently. The reliable pattern is to reach the end of the generator normally, or to wrap manual iteration in a try/finally that calls aclose() explicitly. For early exit from an async for loop, wrap the generator in contextlib.aclosing (Python 3.10+) to call aclose() when the block exits.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.