Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Python Iterable: The Iteration Protocol Explained

Learn what makes an object iterable in Python, how the iteration protocol works, and how to build custom iterables with classes and generators.

~5 min readPython
#Python#Iterators#Generators#Iteration Protocol
Illustration of a Python iterable object feeding items into a for loop

When you write for x in something in Python, the interpreter relies on a contract: something must be an iterable. Understanding that contract, how iteration works under the hood, and how to create your own iterables is a core skill for writing idiomatic Python. This article explains the iteration protocol, the difference between iterables and iterators, and how to build custom iterables both with classes and with generators.

What Is an Iterable in Python?

An iterable is any object that can return an iterator, an object that produces values one at a time. In Python, an object is iterable if it defines __iter__(), or if it defines __getitem__() for sequence-style access. The __getitem__ fallback lets Python iterate over sequence-style objects by calling it with integer indexes starting at 0 and stopping when the object raises IndexError. The for loop, list comprehensions, and many other constructs rely on this protocol.

For example, lists, tuples, dictionaries, sets, strings, and ranges are all iterables. They each implement __iter__() and can be used directly in a for loop:

python
for item in [1, 2, 3]: print(item)

Under the hood, Python calls iter() on the list, which returns an iterator. The loop then repeatedly calls next() on that iterator until StopIteration is raised. This is the essence of the iteration protocol.

The Iteration Protocol: iter and next

The iteration protocol consists of two methods:

  • __iter__(self) returns an iterator object. Iterables implement this method unless they rely on the older __getitem__ sequence-style protocol.
  • __next__(self) returns the next value from the iterator. When no more values are available, it raises StopIteration.

An iterator is an object that implements both __iter__() and __next__(). The __iter__() method of an iterator returns self, which makes the iterator itself iterable.

Here is a minimal iterator that yields numbers from 0 to a limit:

python
class Counter: def __init__(self, limit): self.limit = limit self.value = 0 def __iter__(self): return self def __next__(self): if self.value >= self.limit: raise StopIteration current = self.value self.value += 1 return current

You can use this iterator directly in a for loop:

python
for n in Counter(3): print(n)

This prints 0, 1, 2. The for loop calls iter(Counter(3)), which returns the same object, then calls next() repeatedly until StopIteration is raised.

Common Built-in Iterables

Python's standard library and built-in types provide many iterables. Knowing which types are iterable and how they behave is important for choosing the right data structure.

TypeIterable?Iteration OrderNotes
listYesInsertion orderMutable, allows duplicates
tupleYesInsertion orderImmutable, allows duplicates
dictYesInsertion order (Python 3.7+)Iterates over keys by default
setYesArbitrary (hash-based)Unordered, unique elements
strYesCharacter sequenceIterates over characters
rangeYesNumeric sequenceMemory-efficient for large ranges
bytesYesByte sequenceYields integers

Dictionaries are iterable, but iterating over a dict yields its keys. If you need values or key-value pairs, use .values() or .items(), which return view objects that are also iterable.

python
d = {'a': 1, 'b': 2} for key in d: print(key) # a, b for value in d.values(): print(value) # 1, 2 for key, value in d.items(): print(key, value)

Building a Custom Iterable Class

To make your own class iterable, you need to implement __iter__(). The simplest approach is to make the class its own iterator by also implementing __next__(). However, this pattern has a limitation: the object can only be iterated once. If you want to iterate over the same object multiple times, you need __iter__() to return a fresh iterator each time.

A better pattern is to separate the iterable from the iterator. The iterable defines __iter__() and returns a new iterator object. The iterator holds the state and implements __next__().

python
class RangeIterable: def __init__(self, start, end): self.start = start self.end = end def __iter__(self): return RangeIterator(self.start, self.end) class RangeIterator: def __init__(self, start, end): self.current = start self.end = end def __iter__(self): return self def __next__(self): if self.current >= self.end: raise StopIteration value = self.current self.current += 1 return value

Now you can iterate over the same RangeIterable instance multiple times because each call to iter() creates a new iterator.

python
r = RangeIterable(0, 3) print(list(r)) # [0, 1, 2] print(list(r)) # [0, 1, 2]

Using Generators for Lazy Iteration

Writing full iterator classes is verbose. Python's generator functions provide a concise way to create iterators. A generator function is defined with yield instead of return. When called, it returns a generator object, which is an iterator.

python
def count_up_to(limit): value = 0 while value < limit: yield value value += 1

Using it:

python
for n in count_up_to(3): print(n)

Generators are lazy: they compute values on demand and do not store the entire sequence in memory. This makes them ideal for large or infinite sequences.

You can also use generator expressions, which are similar to list comprehensions but with parentheses:

python
squares = (x * x for x in range(10))

This creates a generator that yields squares one at a time. The generator expression does not allocate a list; it computes each value when requested.

Iterators vs Iterables: How iter() Works

Every iterator is an iterable because it implements __iter__() and returns itself. But not every iterable is an iterator. A list is iterable but not an iterator; it has an __iter__() method but no __next__().

The built-in iter() function is the key. When called on an iterable, it returns an iterator. When called on an iterator, it returns the same object. This is why a for loop works on both.

Consider this example:

python
numbers = [1, 2, 3] iterator = iter(numbers) print(next(iterator)) # 1 print(next(iterator)) # 2

The list itself does not keep track of a current position. The iterator does. Once an iterator is exhausted, calling next() raises StopIteration. You cannot reset an iterator; you must create a new one.

Performance and Memory Considerations

One of the main reasons to understand iterables is to manage memory efficiently. When you process a large dataset, loading it all into a list can consume significant memory. Generators and lazy iterables produce items one at a time, reducing peak memory usage.

For example, reading a large file line by line:

python
with open('huge.log') as f: for line in f: process(line)

The file object is an iterable that yields lines lazily. It does not read the entire file into memory. Similarly, range is lazy; range(10**9) does not allocate a billion integers.

However, lazy iteration is not always free. Each next() call has some overhead, and some operations require random access. If you need to index into a sequence repeatedly, a list or tuple is more appropriate. The choice depends on whether you need the entire collection at once or can process items sequentially.

Common Pitfalls and Edge Cases

A common mistake is to confuse an iterable with an iterator and try to reuse an exhausted iterator. For example:

python
gen = (x for x in range(3)) print(list(gen)) # [0, 1, 2] print(list(gen)) # []

Once a generator is exhausted, it cannot be restarted. If you need to iterate multiple times, create a new generator or use an iterable that returns a fresh iterator each time.

Another pitfall is changing the size of a dictionary while iterating over it. This raises RuntimeError. If you need to delete or add entries, iterate over a copy of its keys:

python
d = {'a': 1, 'b': 2} for key in list(d.keys()): if key == 'a': del d[key]

Finally, be aware that custom iterables that return self from __iter__() are single-use. If you need multi-pass iteration, follow the separate-iterator pattern shown earlier. Understanding these edge cases helps you avoid subtle bugs in production code.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.