Python Cyclic References: How to Detect and Break Them

Python cyclic references occur when two or more objects hold references to each other, forming a cycle. In CPython, reference counting alone cannot collect these cycles, so the cyclic garbage collector must step in. Understanding how these cycles form and how the collector handles them is essential for avoiding memory leaks in long-running Python applications.

What Creates a Cyclic Reference

A cyclic reference appears when objects reference each other directly or indirectly. The simplest case is two objects that point to each other:

class Node:
    def __init__(self, name):
        self.name = name
        self.other = None

a = Node("a")
b = Node("b")
a.other = b
b.other = a

Now a references b and b references a. The reference count of each object is at least 1 because the other object holds a reference. When you delete the external references a and b, the objects still reference each other, so their reference counts never drop to zero. Without a cyclic garbage collector, these objects would leak.

Cycles can also form through containers like lists, dictionaries, or sets, and through objects that hold references in attributes. Even indirect cycles through a chain of objects are possible.

How CPython's Garbage Collector Handles Cycles

CPython uses two mechanisms for memory management: reference counting and a cyclic garbage collector. Reference counting frees objects immediately when their reference count reaches zero. The cyclic collector runs periodically and finds groups of objects that reference each other but are not reachable from the outside.

The collector uses a generational approach. It tracks objects in three generations, with new objects in generation 0. When a generation fills up, the collector runs on that generation and promotes surviving objects to the next generation. The threshold for each generation can be adjusted with gc.set_threshold().

The collector identifies cycles by computing the set of objects that are reachable from a root set. Objects that are part of a cycle and not reachable from any root are collected. This process works for most container types and objects with __dict__, but there are important exceptions.

When Cyclic References Cause Memory Leaks

Most unreachable cycles are collected automatically. A cycle can still cause memory problems if it stays reachable, for example through a global cache or a long-lived object. Another issue is cleanup code that depends on finalizers.

In older CPython versions before Python 3.4, a cycle containing an object with a __del__ method was left uncollected because the interpreter could not decide on a safe order to run the finalizers. Since PEP 442, the cyclic collector can normally collect such cycles after running the finalizers. However, finalizer order is not deterministic, and a finalizer that makes the object reachable again (resurrects it) can prevent collection. Relying on __del__ for cleanup in cyclic object graphs is fragile even in current Python.

Another problematic pattern is holding references to objects that are expensive to create, such as database connections or large caches, inside a cycle. Even if the collector eventually collects the cycle, the objects remain alive longer than necessary, increasing memory pressure.

Detecting Cyclic References with the gc Module

The gc module provides tools to inspect and control the collector. gc.get_objects() returns a list of all objects tracked by the collector. You can filter for objects of a specific type, but this can be slow in production. gc.is_tracked(obj) tells you whether an object is being tracked.

To explore the reference graph, use gc.get_referrers(obj) and gc.get_referents(obj) to walk references in both directions. For a debugging pass, set gc.DEBUG_SAVEALL, call gc.collect(), and inspect gc.garbage. With that flag, the collector keeps unreachable objects in gc.garbage instead of freeing them, which lets you inspect what was part of a cycle.

import gc

gc.set_debug(gc.DEBUG_SAVEALL)
gc.collect()
for obj in gc.garbage:
    print(type(obj), repr(obj))
gc.set_debug(0)
gc.garbage.clear()

Remember to clear gc.DEBUG_SAVEALL and empty gc.garbage after inspection; if you leave the debug flag on, normal collection will keep unreachable objects in memory instead of reclaiming them.

Breaking Cycles with weakref

The weakref module lets you create weak references to objects. A weak reference does not increase the object's reference count, so it does not keep the object alive. When the object is garbage collected, the weak reference returns None when called.

Using weak references is the standard way to break cycles without changing the design of your classes. For example, in a parent-child relationship, the parent can hold a strong reference to the child, but the child should hold a weak reference to the parent.

import weakref

class Parent:
    def __init__(self):
        self.children = []

    def add_child(self, child):
        self.children.append(child)
        child.parent = weakref.ref(self)

class Child:
    def __init__(self, name):
        self.name = name
        self.parent = None

Now the child does not keep the parent alive. When the parent is no longer referenced externally, it can be collected even if the child still exists. If you need to access the parent from the child, call child.parent(); it returns None when the parent has been collected.

Weak references are also useful for caches and callbacks. A callback that holds a strong reference to an object can prevent that object from being collected. Using weakref.ref or weakref.WeakValueDictionary avoids this problem.

Performance and Runtime Considerations

The cyclic collector adds overhead to allocations of objects that the collector tracks. The more tracked objects you create, the more often the collector runs. In performance-critical code, you can adjust the thresholds or disable the collector entirely, but doing so requires that you are certain your code does not create cycles.

gc.disable() stops automatic collection. This can improve performance in short-lived scripts, but it risks memory leaks if cycles are created. You can manually call gc.collect() at safe points, such as after a large batch of work.

The gc.set_threshold(threshold0, threshold1, threshold2) method controls how often each generation is collected. Increasing the thresholds reduces collection frequency but allows more garbage to accumulate. Decreasing them makes collection more aggressive but increases overhead.

For long-running services, it is usually better to keep the collector enabled and rely on weak references to prevent cycles from forming in the first place.

Common Pitfalls with Finalizers and Cycles

The interaction between __del__ and cycles has changed in modern Python, but it is still a common source of bugs. If you can avoid __del__, do so. If you need cleanup code, weakref.finalize is usually a better tool: it registers a callback without relying on __del__, so cyclic collection does not depend on the same finalizer ordering.

When you use weakref.finalize, avoid passing a bound method as the callback. A bound method keeps the instance alive, defeating the purpose. Pass a plain function or static method plus the values you need:

import weakref

class Resource:
    def __init__(self, name):
        self.name = name
        weakref.finalize(self, Resource._cleanup, name)

    @staticmethod
    def _cleanup(name):
        print("Cleaning up", name)

Because the finalizer stores only the static method and the name string, it does not hold a strong reference to the Resource instance.

Another common mistake is storing a strong reference to an object in a global cache without ever removing it. Even if the object is part of a cycle, the cache keeps it alive. Using weakref.WeakValueDictionary for caches prevents this.

Designing Classes to Avoid Unnecessary Cycles

The best way to deal with cyclic references is to avoid creating them when possible. In many cases, you can restructure your object graph to use one-directional references. For example, in a tree the parent can store children in a list, while each child's back-reference to the parent is weak.

When a cycle is inherent to the domain model, use weak references for the back-reference. This is common in observer patterns, where subjects hold a list of observers, and observers hold a weak reference to the subject to avoid keeping it alive.

In practice, you should profile your application's memory usage and use gc.get_objects() to identify unexpected cycles. The tracemalloc module can also help track allocations, but it does not directly show reference cycles.

The key takeaway is that cyclic references are not inherently bad, but they require the garbage collector to work. By understanding when cycles form and using weak references appropriately, you can keep your Python applications memory-efficient and predictable.