Iterators vs iterables
Categories: functional programming
If you have studied Python even for a short while, you will probably have come across the terms iterator, iterable and sequence. They are often used almost interchangeably, but they are all slightly different things.
You might also be interested in the article on iterator protocols.
An iterator is very simple. It is any object that has a
__next__ method. Each time you call the
__next__ method, it will return a value.
For example, the module
itertools contains a function
count that returns an iterator. The particular iterator you get back from
count actually provides a stream of incrementing values 0, 1, 2...
import itertools it = itertools.count() print(it.__next__()) # 0 print(it.__next__()) # 1 print(it.__next__()) # 2...
Python often uses double underscores around a function name to indicate that it is a special function that is important to Python. It makes it less likely that you will accidentally overload it with one of your own functions. However, if you find that code above a bit ugly, you can use the built-in
next function to make it a bit tidier. All
next actually does is call
__next__ on the object you pass in. Here is the code using
next, it does exactly the same thing:
import itertools it = itertools.count() print(next(it)) # 0 print(next(it)) # 1 print(next(it)) # 2...
iterable is an object that you can iterate over. A list is an example of an iterable (so are strings, tuples and range objects).
iterator is an object that has and
__iter__ method. The
__iter__ method returns an
iterator can be used to get the items in the list, one by one, using the
You rarely need to worry about these details. The most common way to iterate over an iterable is in a for loop:
k = [10, 20, 30, 40] for x in k: print(x)
k is the iterable. The for loop reads the values from the iterable, one at a time, and executes the loop for each value.
for loops under the hood
Now we will take a look at what a for loop actually does. First, we need to get the
iterator from the
iterable. As we saw above, you use the
__iter__ method to do this, but a less ugly alternative is to use the
iter function (which just calls the
k = [10, 20, 30, 40] it = iter(k)
Now you can use the
next function to read the values of the original list via the iterator, one by one:
print(next(it)) print(next(it)) print(next(it)) print(next(it)) print(next(it)) // StopIteration exception
Each time you call
next on the iterator
it, it fetches the next value from the iterable (the list
k). When you reach the end of the iterable,
next will throw a
StopIteration exception. This tells Python that the iterable
k has no more values left.
You might wonder why Python throws an exception, rather than providing a function you can call to check if you are at the end of the list. Well, in some cases it isn't possible to know whether you are at the end of the sequence until you actually try to calculate the next value (we will see an example later). Since our iterator doesn't calculate the next value until it is asked to, an exception is the best option.
Don't worry, you will rarely write code like this, you will almost always use for to do the work. In summary, here is what a for loop does when you set it running on an iterable:
iterto get the iterator for the iterable
nextrepeatedly on the iterator, executing the loop each time
- Catches the
StopIterationexception and ends the loop
Looping over an iterator
If you recall, and iterator has a
__next__ method, and an iterable has an
However, every iterator is also an iterable. That is, iterables don't just have an
__next__ method, they have an
__iter__ method too!
That means you can call
iter on an iterator to find its ... iterator. Since it is already an iterator, it just returns itself!
This might seem a bit odd, but it is actually very useful. It allows you to use a for loop with either an iterable or an iterator.
A sequence is type of iterable that also provides random access to elements. A sequence has some extra methods, for example
__setitem__. Python uses these low level methods to provide various language features, for example:
k = [10, 20, 30, 40] # A list is a sequence x = k # uses __getitem__ len(k) # uses __len__ k = 0 # uses __setitem__
For more information see the Python documentation at python.org
This diagram shows the how iterables, iterators and sequences are related. They are all, ultimately, iterables, and any iterable can
create an iterator using the
You don't normally need to implement iterators at this low level, you can usually use a generator.
- Introduction to Functional Programming
- Pure functions
- Lambda functions
- Built-in functions on iterables
- Transforming iterables
- Map/reduce example
- Functional design patterns
- Recursion and the lru_cache in Python
- Partial application
- Failure monad
- List monad
- Maybe monad
Join the PythonInformer Newsletter
Sign up using this form to receive an email when new content is added:
2d arrays abstract data type alignment and angle animation arc array arrays bar chart bar style behavioural pattern bezier curve built-in function callable object chain circle classes clipping close closure cmyk colour combinations comparison operator comprehension context context manager conversion count creational pattern data science data types decorator design pattern device space dictionary drawing duck typing efficiency ellipse else encryption enumerate fill filter font font style for loop formula function function composition function plot functools game development generativepy tutorial generator geometry gif global variable gradient greyscale higher order function hsl html image image processing imagesurface immutable object in operator index inner function input installing iter iterable iterator itertools join l system lambda function latex len lerp line line plot line style linear gradient linspace list list comprehension logical operator lru_cache magic method mandelbrot mandelbrot set map marker style matplotlib monad mutability named parameter numeric python numpy object open operator optimisation optional parameter or pandas partial application path pattern permutations pie chart pil pillow polygon pong positional parameter print product programming paradigms programming techniques pure function python standard library radial gradient range recipes rectangle recursion reduce regular polygon repeat rgb rotation roundrect scaling scatter plot scipy sector segment sequence setup shape singleton slice slicing sound spirograph sprite square str stream string stroke structural pattern subpath symmetric encryption template tex text text metrics tinkerbell fractal transform translation transparency triangle truthy value tuple turtle unpacking user space vectorisation webserver website while loop zip zip_longest