C# Iterator Deferred Execution

When you write an iterator method that uses yield return, the method body does not run when you call it. The call returns an IEnumerable<T> that runs only when something starts enumerating it, usually with foreach or by calling GetEnumerator() and MoveNext(). That behavior is called deferred execution, and it changes when exceptions and side effects occur, and it has performance and memory tradeoffs.

What Deferred Execution Means for Iterator Methods

When you define a method that uses yield return, calling that method does not execute the method body. Instead, it returns an IEnumerable<T> object that represents the sequence. The code inside the iterator runs only when something starts enumerating the sequence, typically with foreach or GetEnumerator() and MoveNext(). This is the defining behavior of iterator methods in C#.

Consider this example:

public static IEnumerable<int> GetNumbers()
{
    Console.WriteLine("Iterator started");
    yield return 1;
    Console.WriteLine("After first yield");
    yield return 2;
    Console.WriteLine("After second yield");
}

If you call var numbers = GetNumbers();, nothing is printed. The Console.WriteLine calls execute only when you iterate:

foreach (var n in numbers)
{
    Console.WriteLine($"Got {n}");
}

The output order shows that the iterator resumes after each yield return. This deferred behavior is by design and is the foundation of lazy sequences in C#.

How the Compiler Turns Iterators into State Machines

For an iterator method whose return type is IEnumerable<T>, the C# compiler transforms the method into a generated class that implements both IEnumerable<T> and IEnumerator<T>. The local variables that must survive across yield boundaries become fields of that class, and the code between yield statements becomes a state machine with a MoveNext() method. Each call to MoveNext() executes the next chunk of code until it hits another yield return or the method ends.

This transformation is why you cannot use ref or out parameters in iterator methods, and why yield return cannot appear inside a try block with a catch. The generated state machine must preserve the exact position between calls.

Understanding this mechanism explains why the iterator body does not run at the time of the method call: the call creates the state machine object, and the first call to MoveNext() runs the code from the beginning until the first yield return.

When the Iterator Body Actually Runs

The iterator body starts executing on the first call to MoveNext(), not when the method is invoked. This timing affects when exceptions are thrown. If an iterator method throws an exception before the first yield return, that exception is not thrown when you call the method; it is thrown when you start enumerating.

For example:

public static IEnumerable<int> GetNumbers()
{
    throw new InvalidOperationException("Boom");
    yield return 1;
}

Calling GetNumbers() succeeds and returns an enumerable. The exception is thrown only when you call GetEnumerator().MoveNext() or use foreach. This differs from a regular method, where the exception would be thrown immediately.

This behavior matters for APIs that return sequences. A caller might assume that calling the method validates arguments, but with deferred execution, validation must be done separately if you need it to happen at call time.

Side Effects and Exception Timing

Because the iterator body executes lazily, any side effects it performs—writing to a log, modifying a shared field, or incrementing a counter—happen during enumeration, not at the method call. If you enumerate the same sequence twice, the side effects happen twice. This is a common source of bugs.

Consider a method that reads from a file or a database. If you return an iterator that reads rows lazily, the file or connection is opened only when you start iterating. If you iterate multiple times, you may get different results when the underlying data changes. This is often desirable for streaming, but it also means you must handle resource lifetime explicitly.

Performance and Memory Implications

Deferred execution allows you to build pipelines that avoid materializing intermediate collections. For example, you can chain LINQ operations like Where, Select, and Take without creating new arrays or lists until the final result is consumed. This can reduce memory usage and avoid copies for large sequences.

Deferred execution also has costs. Each MoveNext() call involves a state machine transition, which has some overhead compared to a simple loop over an array. For small sequences, the overhead is negligible, but for hot paths that enumerate millions of items, it can matter. Also, if you enumerate the same iterator multiple times, the code runs again, which can be expensive.

The key tradeoff is between memory and CPU. Materializing with ToList() gives you a snapshot and allows repeated enumeration without re-executing the iterator, but it allocates memory. Deferred execution keeps memory low but may re-run the iterator logic.

Forcing Execution with ToList and ToArray

When you need to execute the iterator immediately, you can call ToList() or ToArray() on the enumerable. This forces the iterator to run to completion and stores the results in a collection. After that, the collection is independent of the original iterator.

var numbers = GetNumbers().ToList();

Now the iterator has run once, and numbers is a List<int> that you can enumerate multiple times without re-running the iterator logic. This is useful when you need a snapshot of the data or when the iterator has side effects that should happen only once.

Be careful with infinite sequences: calling ToList() on an infinite sequence will never return. Deferred execution lets you take only the first few items with Take, but materializing the whole sequence is impossible.

Common Pitfalls with Multiple Enumeration

A frequent mistake is enumerating an iterator multiple times without realizing it. If you pass an IEnumerable<T> to a method that iterates it twice, the iterator runs twice. If the iterator is expensive or has side effects, this can cause unexpected behavior.

public static void Process(IEnumerable<int> items)
{
    var count = items.Count();
    foreach (var item in items)
    {
        // ...
    }
}

If items is a deferred iterator, Count() forces the first enumeration, and the foreach forces a second. You can avoid this by materializing the sequence once at the start of the method.

Deferred Execution in LINQ and Custom Iterators

LINQ operators like Where and Select are implemented as iterator methods, so they defer execution. Other operators, such as OrderBy, are also deferred even though they use a different implementation. When you write var query = source.Where(x => x > 5);, the query is not executed until you enumerate it. This lets you compose a query before consuming it. The query captures the source and predicate references at the time it is created; if the source is a live collection, you may observe changes to its contents during enumeration.

Custom iterators follow the same rules. If you implement a method that returns IEnumerable<T> using yield, you inherit deferred execution behavior. This is useful for creating streaming data sources, but you should document the behavior so callers know when side effects occur.

One edge case: if you use yield break, the iterator ends without yielding any items. The method body still does not run until enumeration, and yield break simply signals the end of the sequence.