Python Asyncio: Event Loop and Tasks Explained
How the asyncio event loop schedules coroutines and tasks in Python, with concurrency patterns, TaskGroup, cancellation, timeouts, and the failure modes that break production services.

Python asyncio is a single-threaded cooperative scheduler: one event loop owns a queue of ready callbacks, and every async def function you await yields control back to that loop instead of blocking the OS thread. Concurrency comes from overlapping waits, not from running Python bytecode in parallel.
The event loop is a scheduler, not a thread pool
An event loop is a while loop around a selector. It asks the operating system which file descriptors are ready, runs the callbacks registered for them, and then runs whatever else is queued. Nothing inside a coroutine preempts anything else, so a coroutine that never awaits stops the entire loop.
import asyncio
async def fetch(name: str, delay: float) -> str:
await asyncio.sleep(delay) # yields control to the loop
return f"{name} done"
async def main() -> None:
loop = asyncio.get_running_loop()
print(loop.time())
results = await asyncio.gather(
fetch("a", 0.2),
fetch("b", 0.1),
)
print(results)
asyncio.run(main())asyncio.run() builds a fresh event loop for the current thread, runs the coroutine until it completes, cancels anything still pending, shuts down async generators, and closes the loop. Call it once at the top of a program. Calling it from inside an already running loop raises RuntimeError, and asyncio.get_event_loop() outside a running loop is deprecated as of Python 3.12, so use asyncio.get_running_loop() inside coroutines.
Coroutines, tasks, and futures
A coroutine object does nothing until it is awaited or scheduled. A Task wraps a coroutine and schedules it on the loop immediately, which is what makes concurrency possible.
async def main() -> None:
coro = fetch("a", 0.2) # nothing has run yet
task = asyncio.create_task(coro) # scheduled now
await asyncio.sleep(0) # let the task start
print(task.done()) # False
print(await task)| Object | Scheduled immediately | Awaitable | Result access | Typical use |
|---|---|---|---|---|
| Coroutine object | No | Yes | Only through await | Building a call to schedule later |
Task | Yes | Yes | await, .result() after done | Fire concurrent work on the loop |
Future | No (low-level) | Yes | set_result, await | Bridging callbacks to async code |
| Thread | Yes (OS) | No | join, queue | Blocking or CPU-bound work |
The distinction matters because forgetting it is the most common asyncio bug in production code: calling an async function without awaiting returns a coroutine object, a RuntimeWarning: coroutine was never awaited appears on garbage collection, and the work silently never happens.
Concurrency patterns that hold up
asyncio.gather runs awaitables concurrently and returns results in input order. It is the workhorse for fan-out calls to independent services.
async def load_all(ids: list[int]) -> list[dict]:
async with httpx.AsyncClient() as client:
tasks = [client.get(f"https://api.example.com/items/{i}") for i in ids]
responses = await asyncio.gather(*tasks, return_exceptions=False)
return [r.json() for r in responses]return_exceptions=True converts failures into values in the result list instead of raising the first exception. That is a deliberate tradeoff: it never loses a result, but it also hides failures unless you inspect every element.
Since Python 3.11, asyncio.TaskGroup gives structured concurrency. If any child task raises, the group cancels its siblings and re-raises the failures as an ExceptionGroup.
async def main() -> None:
async with asyncio.TaskGroup() as tg:
a = tg.create_task(fetch("a", 0.2))
b = tg.create_task(fetch("b", 0.1))
print(a.result(), b.result())Use TaskGroup for new code. The async with block cannot be exited until every child has finished or been cancelled, so you cannot leak a task by accident, and cancellation propagates inward instead of being swallowed.
Deep dive: the failure modes that only appear under load
1. Blocking calls inside a coroutine
Any synchronous call that blocks - time.sleep, requests.get, a large json.loads on a slow disk, input() - blocks the loop and therefore every other task. Offload it with asyncio.to_thread, which runs the function in the loop's default executor.
import asyncio, time
def parse_big_file(path: str) -> int:
time.sleep(2) # simulates CPU/IO heavy work
return 42
async def main() -> None:
# Bad: blocks the loop for two seconds
# parse_big_file("data.csv")
value = await asyncio.to_thread(parse_big_file, "data.csv")
print(value)to_thread is for code you cannot rewrite. It still costs an OS thread per call, so a burst of thousands of such calls exhausts the default executor. For pure computation, asyncio is the wrong tool; use multiprocessing or a worker process.
2. Fire-and-forget tasks that get garbage collected
The event loop keeps only weak references to tasks. A task created without keeping a reference can disappear before it finishes, producing the classic message Task was destroyed but it is pending!.
_background: set[asyncio.Task] = set()
def spawn(coro) -> None:
task = asyncio.create_task(coro)
_background.add(task)
task.add_done_callback(_background.discard)Keeping a strong reference is not optional. Alternatively, asyncio.TaskGroup removes the whole category of bug by owning the lifetime of its children.
3. Cancellation is not optional cleanup
asyncio.CancelledError inherits from BaseException since Python 3.8. A bare except Exception will not catch it, which is intentional: cancellation must always propagate. If you catch it for cleanup, re-raise it.
async def worker(queue: asyncio.Queue) -> None:
try:
while True:
item = await queue.get()
await handle(item)
except asyncio.CancelledError:
await flush_partial_work()
raise # never swallow cancellation
finally:
queue.task_done() if not queue.empty() else Noneasyncio.shield() protects an inner awaitable from the outer cancellation, but the shield itself still raises, so the protected work continues while the caller learns that it was cancelled.
4. Unbounded concurrency
Gathering 10,000 requests at once is not concurrency, it is a denial-of-service attack on your own dependencies. Bound it with a semaphore, or with a fixed number of workers reading from a queue.
async def bounded(items: list[str], limit: int = 20) -> list[str]:
sem = asyncio.Semaphore(limit)
async def one(item: str) -> str:
async with sem:
return await fetch(item, 0.1)
return await asyncio.gather(*(one(i) for i in items))The semaphore admits limit tasks at a time; the rest wait without consuming a thread. Set the limit from the downstream service's capacity, not from a guess.
5. Timeouts and the loop's own health
asyncio.wait_for and, since 3.11, asyncio.timeout, wrap a deadline around an awaitable. Both cancel the inner task on expiry, so the inner code must be cancellation-safe.
async def with_deadline() -> str:
try:
async with asyncio.timeout(1.5):
return await fetch("slow", 5.0)
except TimeoutError:
return "fallback"Enable asyncio's debug mode in tests (PYTHONASYNCIODEBUG=1 or asyncio.run(main(), debug=True)). It logs every coroutine that took longer than loop.slow_callback_duration (100 ms by default), plus un-awaited coroutines and un-retrieved task exceptions, which is usually the fastest way to find the blocking call that froze the loop.
Common mistakes
- Calling an async function without
await. You get a coroutine object, a warning at collection time, and no work done. - Using
time.sleepinside a coroutine. It blocks the loop, not just the task. Useawait asyncio.sleep(). - Mixing threads and loops. A loop belongs to one thread. Use
asyncio.run_coroutine_threadsafe()to submit work from another thread instead of touching the loop directly. - Assuming
gatherstops on error. With the defaultreturn_exceptions=False, the first exception is raised in the caller, but sibling tasks keep running until they finish or are cancelled. - Ignoring slow callbacks. Debug mode reports them; in production they show up as latency spikes that no amount of scaling hides.
- Using asyncio for CPU-bound work. The GIL and the single-threaded loop make it slower than a plain loop plus processes.
The mental model that prevents most of these: the loop is a scheduler with one thread, await is the only place control changes hands, and every task must have an owner responsible for its result and its exceptions.
Try matching browser tools
All toolsSmall utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Python guides
All Python guidesPython iter(): How the Built-in Function Works
Explains Python's iter() built-in: how it creates iterators, the two-argument sentinel form, custom iterator classes, and common edge cases.
Python Pika: Queues, Exchanges, Routing Keys, and Acknowledgements
Learn how to use Python's Pika client with RabbitMQ: declare queues and exchanges, publish with routing keys, consume messages, and handle acknowledgements reliably.
Python Kafka: kafka-python vs Confluent Client
python kafka python vs confluent kafka: Compare kafka-python and Confluent Kafka Python clients: API differences, performance, feature coverage, and when to choose eac...
Python Kafka Offsets, Partitions, and Manual Commits
Learn how Kafka offsets and partitions work and how to manually commit offsets in Python consumers with kafka-python and confluent-kafka. Covers sync vs async commits, rebalances, and seeking.
More in Python
All Python guidesThe guides filed next to this one in Python, in library order.