Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Python Asyncio: Event Loop and Tasks Explained

How the asyncio event loop schedules coroutines and tasks in Python, with concurrency patterns, TaskGroup, cancellation, timeouts, and the failure modes that break production services.

~5 min readPython
#Python#Asyncio#Concurrency#Event Loop
Python asyncio event loop with tasks connected in a ring

Python asyncio is a single-threaded cooperative scheduler: one event loop owns a queue of ready callbacks, and every async def function you await yields control back to that loop instead of blocking the OS thread. Concurrency comes from overlapping waits, not from running Python bytecode in parallel.

The event loop is a scheduler, not a thread pool

An event loop is a while loop around a selector. It asks the operating system which file descriptors are ready, runs the callbacks registered for them, and then runs whatever else is queued. Nothing inside a coroutine preempts anything else, so a coroutine that never awaits stops the entire loop.

python
import asyncio async def fetch(name: str, delay: float) -> str: await asyncio.sleep(delay) # yields control to the loop return f"{name} done" async def main() -> None: loop = asyncio.get_running_loop() print(loop.time()) results = await asyncio.gather( fetch("a", 0.2), fetch("b", 0.1), ) print(results) asyncio.run(main())

asyncio.run() builds a fresh event loop for the current thread, runs the coroutine until it completes, cancels anything still pending, shuts down async generators, and closes the loop. Call it once at the top of a program. Calling it from inside an already running loop raises RuntimeError, and asyncio.get_event_loop() outside a running loop is deprecated as of Python 3.12, so use asyncio.get_running_loop() inside coroutines.

Coroutines, tasks, and futures

A coroutine object does nothing until it is awaited or scheduled. A Task wraps a coroutine and schedules it on the loop immediately, which is what makes concurrency possible.

python
async def main() -> None: coro = fetch("a", 0.2) # nothing has run yet task = asyncio.create_task(coro) # scheduled now await asyncio.sleep(0) # let the task start print(task.done()) # False print(await task)
ObjectScheduled immediatelyAwaitableResult accessTypical use
Coroutine objectNoYesOnly through awaitBuilding a call to schedule later
TaskYesYesawait, .result() after doneFire concurrent work on the loop
FutureNo (low-level)Yesset_result, awaitBridging callbacks to async code
ThreadYes (OS)Nojoin, queueBlocking or CPU-bound work

The distinction matters because forgetting it is the most common asyncio bug in production code: calling an async function without awaiting returns a coroutine object, a RuntimeWarning: coroutine was never awaited appears on garbage collection, and the work silently never happens.

Concurrency patterns that hold up

asyncio.gather runs awaitables concurrently and returns results in input order. It is the workhorse for fan-out calls to independent services.

python
async def load_all(ids: list[int]) -> list[dict]: async with httpx.AsyncClient() as client: tasks = [client.get(f"https://api.example.com/items/{i}") for i in ids] responses = await asyncio.gather(*tasks, return_exceptions=False) return [r.json() for r in responses]

return_exceptions=True converts failures into values in the result list instead of raising the first exception. That is a deliberate tradeoff: it never loses a result, but it also hides failures unless you inspect every element.

Since Python 3.11, asyncio.TaskGroup gives structured concurrency. If any child task raises, the group cancels its siblings and re-raises the failures as an ExceptionGroup.

python
async def main() -> None: async with asyncio.TaskGroup() as tg: a = tg.create_task(fetch("a", 0.2)) b = tg.create_task(fetch("b", 0.1)) print(a.result(), b.result())

Use TaskGroup for new code. The async with block cannot be exited until every child has finished or been cancelled, so you cannot leak a task by accident, and cancellation propagates inward instead of being swallowed.

Deep dive: the failure modes that only appear under load

1. Blocking calls inside a coroutine

Any synchronous call that blocks - time.sleep, requests.get, a large json.loads on a slow disk, input() - blocks the loop and therefore every other task. Offload it with asyncio.to_thread, which runs the function in the loop's default executor.

python
import asyncio, time def parse_big_file(path: str) -> int: time.sleep(2) # simulates CPU/IO heavy work return 42 async def main() -> None: # Bad: blocks the loop for two seconds # parse_big_file("data.csv") value = await asyncio.to_thread(parse_big_file, "data.csv") print(value)

to_thread is for code you cannot rewrite. It still costs an OS thread per call, so a burst of thousands of such calls exhausts the default executor. For pure computation, asyncio is the wrong tool; use multiprocessing or a worker process.

2. Fire-and-forget tasks that get garbage collected

The event loop keeps only weak references to tasks. A task created without keeping a reference can disappear before it finishes, producing the classic message Task was destroyed but it is pending!.

python
_background: set[asyncio.Task] = set() def spawn(coro) -> None: task = asyncio.create_task(coro) _background.add(task) task.add_done_callback(_background.discard)

Keeping a strong reference is not optional. Alternatively, asyncio.TaskGroup removes the whole category of bug by owning the lifetime of its children.

3. Cancellation is not optional cleanup

asyncio.CancelledError inherits from BaseException since Python 3.8. A bare except Exception will not catch it, which is intentional: cancellation must always propagate. If you catch it for cleanup, re-raise it.

python
async def worker(queue: asyncio.Queue) -> None: try: while True: item = await queue.get() await handle(item) except asyncio.CancelledError: await flush_partial_work() raise # never swallow cancellation finally: queue.task_done() if not queue.empty() else None

asyncio.shield() protects an inner awaitable from the outer cancellation, but the shield itself still raises, so the protected work continues while the caller learns that it was cancelled.

4. Unbounded concurrency

Gathering 10,000 requests at once is not concurrency, it is a denial-of-service attack on your own dependencies. Bound it with a semaphore, or with a fixed number of workers reading from a queue.

python
async def bounded(items: list[str], limit: int = 20) -> list[str]: sem = asyncio.Semaphore(limit) async def one(item: str) -> str: async with sem: return await fetch(item, 0.1) return await asyncio.gather(*(one(i) for i in items))

The semaphore admits limit tasks at a time; the rest wait without consuming a thread. Set the limit from the downstream service's capacity, not from a guess.

5. Timeouts and the loop's own health

asyncio.wait_for and, since 3.11, asyncio.timeout, wrap a deadline around an awaitable. Both cancel the inner task on expiry, so the inner code must be cancellation-safe.

python
async def with_deadline() -> str: try: async with asyncio.timeout(1.5): return await fetch("slow", 5.0) except TimeoutError: return "fallback"

Enable asyncio's debug mode in tests (PYTHONASYNCIODEBUG=1 or asyncio.run(main(), debug=True)). It logs every coroutine that took longer than loop.slow_callback_duration (100 ms by default), plus un-awaited coroutines and un-retrieved task exceptions, which is usually the fastest way to find the blocking call that froze the loop.

Common mistakes

  • Calling an async function without await. You get a coroutine object, a warning at collection time, and no work done.
  • Using time.sleep inside a coroutine. It blocks the loop, not just the task. Use await asyncio.sleep().
  • Mixing threads and loops. A loop belongs to one thread. Use asyncio.run_coroutine_threadsafe() to submit work from another thread instead of touching the loop directly.
  • Assuming gather stops on error. With the default return_exceptions=False, the first exception is raised in the caller, but sibling tasks keep running until they finish or are cancelled.
  • Ignoring slow callbacks. Debug mode reports them; in production they show up as latency spikes that no amount of scaling hides.
  • Using asyncio for CPU-bound work. The GIL and the single-threaded loop make it slower than a plain loop plus processes.

The mental model that prevents most of these: the loop is a scheduler with one thread, await is the only place control changes hands, and every task must have an owner responsible for its result and its exceptions.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.