Python Iterators and the Iterator Protocol Explained

Python iterators, explained

Every for loop, comprehension, and sum() in Python rests on one small protocol. Once you see the machinery, a lot of behaviour — why a generator is "used up", why zip returns something you can only loop once — suddenly makes sense. This guide builds it up from iter() and next().

Iterable vs iterator

These two words sound the same but mean different things:

An iterable is anything you can loop over — a list, string, dict, file. It implements __iter__, which returns an iterator.
An iterator is the object that actually produces values one at a time. It implements __next__ (and returns itself from __iter__).

nums = [1, 2, 3]        # iterable (a list)
it = iter(nums)         # iterator
type(it)                # <class 'list_iterator'>

A list is iterable but is not its own iterator — you get a fresh iterator each time you call iter() on it.

iter() and next()

iter(obj) gets an iterator; next(it) pulls the next value. When values run out, next raises StopIteration:

it = iter([10, 20])
next(it)        # 10
next(it)        # 20
next(it)        # StopIteration

next(it, "done")   # supply a default to avoid the exception -> 'done'

What a for loop actually does

A for loop is sugar for: call iter() on the iterable, repeatedly call next(), and stop when StopIteration is raised. These are equivalent:

for x in [1, 2, 3]:
    print(x)

# desugared:
it = iter([1, 2, 3])
while True:
    try:
        x = next(it)
    except StopIteration:
        break
    print(x)

This is why anything implementing the protocol works in a for loop, in test, comprehension, sum, max, unpacking, and so on.

Iterators are exhausted after one pass

An iterator holds its position and moves forward only — once consumed, it's empty. This is the source of many "my data disappeared" bugs:

it = iter([1, 2, 3])
list(it)       # [1, 2, 3]
list(it)       # []  — already exhausted!

The same applies to generators, zip, map, and filter objects. If you need to iterate twice, keep the underlying list, or call iter() again on the original iterable (not on the spent iterator).

Building a custom iterator

To make a class iterable, implement __iter__ (return the iterator) and __next__ (produce values or raise StopIteration):

class Countdown:
    def __init__(self, start):
        self.n = start
    def __iter__(self):
        return self                 # the object is its own iterator
    def __next__(self):
        if self.n <= 0:
            raise StopIteration
        self.n -= 1
        return self.n + 1

list(Countdown(3))   # [3, 2, 1]

Returning self from __iter__ makes this single-use, like the built-in iterators.

Generators: iterators without the boilerplate

Writing __iter__/__next__ by hand is rarely necessary — a generator function (any function with yield) builds an iterator for you, managing state and StopIteration automatically:

def countdown(start):
    while start > 0:
        yield start
        start -= 1

list(countdown(3))   # [3, 2, 1]

This is the idiomatic way to produce custom iteration; the class form is mostly useful when you need extra methods or attributes alongside iteration.

Recap

An iterable has __iter__ and can be looped over repeatedly; an iterator has __next__, yields values once, and raises StopIteration when done. A for loop is sugar for iter() then repeated next() until StopIteration. Iterators (including generators, zip, map, filter) are single-use — re-create them to iterate again. Implement the protocol with __iter__/__next__ for a custom iterator, but reach for a generator function when you just want iteration without the boilerplate.

Python Iterators and the Iterator Protocol Explained — iter, next, and for Loops