Dan Bader

Comprehending Python’s Comprehensions

One of my favorite features in Python are list comprehensions. They can seem a bit arcane at first but when you break them down they are actually a very simple construct.

Comprehending Python's List, Dict, Set Comprehensions

The key to understanding list comprehensions is that they’re just for-loops over a collection expressed in a more terse and compact syntax. Let’s take the following list comprehension as an example:

>>> squares = [x * x for x in range(10)]

It computes a list of all integer square numbers from 0 to 9:

>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

If we wanted to build the same list using a plain for-loop we’d probably write something like this:

>>> squares = []
>>> for x in range(10):
...    squares.append(x * x)

That’s a pretty straightforward loop, right? If you try and generalize some of this structure you might end up with a template similar to this:

(values) = [ (expression) for (value) in (collection) ]

The above list comprehension is equivalent to the following plain for-loop:

(values) = []
for (value) in (collection):
    (values).append( (expression) )

Again, a fairly simple cookiecutter pattern you can apply to most for loops. Now there’s one more useful element we need to add to this template, and that is element filtering with conditions.

List comprehensions can filter values based on some arbitrary condition that decides whether or not the resulting value becomes a part of the output list. Here’s an example:

>>> even_squares = [x * x for x in range(10)
                    if x % 2 == 0]

This list comprehension will compute a list of the squares of all even integers from 0 to 9.

If you’re not familiar with what the modulo (%) operator does—it returns the remainder after division of one number by another. In this example the %-operator gives us an easy way to test if a number is even by checking the remainder after we divide the number by 2.

>>> even_squares
[0, 4, 16, 36, 64]

Similarly to the first example, this new list comprehension can be transformed into an equivalent for-loop:

even_squares = []
for x in range(10):
    if x % 2 == 0:
        even_squares.append(x)

Let’s try and generalize the above list comprehension to for-loop transform again. This time we’re going to add a filter condition to our template to decide which values end up in the resulting list.

Here’s the list comprehension template:

values = [expression
          for value in collection
          if condition]

And we can transform this list comprehension into a for-loop with the following pattern:

vals = []
for value in collection:
    if condition:
        vals.append(expression)

Again, this is a straightforward transformation—we simply apply our cookiecutter pattern again. I hope this dispelled some of the “magic” in how list comprehensions work. They’re really quite a useful tool.

Before you move on I want to point out that Python not only supports list comprehensions but also has similar syntax for sets and dictionaries.

Here’s what a set comprehension looks like:

>>> { x * x for x in range(-9, 10) }
set([64, 1, 36, 0, 49, 9, 16, 81, 25, 4])

And this is a dict comprehension:

>>> { x: x * x for x in range(5) }
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}

Both are useful tools in practice. There’s one caveat to Python’s comprehensions—as you get more proficient at using them it becomes easier and easier to write code that’s difficult to read. If you’re not careful you might have to deal with monstrous list, set, dict comprehensions soon. Remember, too much of a good thing is usually a bad thing.

After much chagrin I’m personally drawing the line at one level of nesting for comprehensions. I found that in most cases it’s better (as in “more readable” and “easier to maintain”) to use for-loops beyond that point.

📘 Python Tricks: The Book — A Buffet of Awesome Python Features: My new book that teaches you the coolest aspects of Python with short and easy to digest examples. » Click here to get a free sample chapter

Improve Your Python with a fresh 🐍 Python Trick 💌 every couple of days

🔒 No spam ever. Unsubscribe any time.

This article was filed under: programming, and python.

Related Articles:
Latest Articles:
  • Context Managers and the “with” Statement in Python – The “with” statement in Python is regarded as an obscure feature by some. But when you peek behind the scenes of the underlying Context Manager protocol you’ll see there’s little “magic” involved.
  • Installing Python and Pip on Windows – In this tutorial you’ll learn how to set up Python and the Pip package manager on Windows 10, completely from scratch.
  • Sublime Text Settings for Writing Clean Python – How to write beautiful and clean Python by tweaking your Sublime Text settings so that they make it easier to adhere to the PEP 8 style guide recommendations.
  • How Sublime Text’s Preferences Work – Sublime Text uses a powerful text-based settings system that can be a little intimidating when you encounter it for the first time. This article gives an overview of the system and points out some common mistakes to avoid.
  • Writing Clean Python With Namedtuples – Python comes with a specialized “namedtuple” container type that doesn’t seem to get the attention it deserves. It’s one of these amazing features in Python that’s hidden in plain sight.
  • Lambda Functions in Python: What Are They Good For? – An introduction to “lambda” expressions in Python: What they’re good for, when you should use them, and when it’s best to avoid them.
← Browse All Articles