How to Structure Your Python Programs

How should you structure your Python programs?

Is there a recommended way to bring “structure to the chaos?”

If you’re writing anything that goes beyond a few lines of Python code, how should you lay out your functions and other building blocks?

Interesting thought on that topic from a newsletter member:

I have always felt that programming is telling a computer a story.

If your story is good, the computer will execute it efficiently, if the story sucks, the execution sucks.

I write code like I would write a technical paper, it has direction and flow, and you can tell when you have reached the conclusion of the story.

Cute! And true:

I’ve had good results with this “narrative” approach. It works especially well for single-file automation or data crunching scripts. And it helps you keep your code organized and maintainable, even as a project grows.

Let’s take a look at how this would work in practice. We’ll first lay out the logical flow for an example program and then we’ll compare different ways to implement this narrative in Python.

Breaking Down the “Program Narrative”

Imagine the following high-level logical flow for a simple report generator program:

1. Read input data
2. Perform calculations
3. Write report

Notice how each stage (after the first one) depends on some byproduct or output of its predecessor:

1. Read input data
2. Perform calculations (based on input data)
3. Write report (based on calculated report data)

The way I see it, you have two choices here: You can either implement this logical flow from the top down or from the bottom up.

“Top-Down” vs “Bottom-Up” Code Layout

If write your program bottom-up, your function layout will match the logic flow—it’ll go from the fully independent building blocks to the ones that depend on their results.

Here’s a sketch for a “bottom-up” implementation:

def read_input_file(filename):
    pass

def generate_report(data):
    pass

def write_report(report):
    pass

data = read_input_file('data.csv')
report = generate_report(data)
write_report(report)

This structure “makes sense” in an intuitive way, doesn’t it?

We first need to read the input file before we can generate a report, and we need to generate the report before we can write it out to disk.

This logical structure is reflected in the program’s layout.

[Or, in scary-sounding Computer Science terms: This is basically a topological sorting of the dependency graph.]

Let’s take a look at the “top-down” implementation:

For a “top-down” approach you’d flip the same structure on its head and start with the highest-level building block first, fleshing out the details later.

This results in the following program sketch:

def main():
    data = read_input_file('data.csv')
    report = generate_report(data)
    write_report(report)

def write_report(report):
    pass

def generate_report(data):
    pass

def read_input_file(filename):
    pass

# Application entry point -> call main()
main()

See how I started with the high-level, “most dependent” functionality first this time around?

The “main()” function at the top states clearly what this program is going to do—without having defined yet how exactly it will achieve the desired result.

Which Approach Is Better:
“Top-Down” or “Bottom-Up?”

I don’t think there’s much of a practical difference between them, to be honest.

The important thing for me is that they both encode a logical narrative—they’re both “telling the computer a story” and they have “direction and flow.”

This is the key insight for me.

The worst thing one can do is deliberately obfuscating this logical structure, thereby killing the narrative:

def write_report(report):
    pass

def read_input_file(filename):
    pass

def generate_report(data):
    pass

(Yuck!)

Now, obviously I’m using a small “toy” example here—

But imagine what happens with programs that consist of 10, 100, 1000 steps in their “narrative” and they’re organized incoherently?

Maybe it’s my German need for order and stability speaking—but in my experience the result is usually utter chaos and madness:

“If the story sucks, the execution sucks”

The more you practice this “narrative flow” mindset as a way of structuring your programs, the more natural it will feel and the more automatic it will become as a behavior while you’re coding.

If you’re looking for a way to practice this method, try to revisit some of your older Python code and rewrite/refactor it to follow the principles laid out in this article.

Of course, you can also extend this idea to other “building blocks” like classes and modules as well…but more on that at some other time.

Happy Pythoning!