Writing Python Command-Line Tools With Click

Python is often referred to as a glue code language because it’s extremely flexible and integrates well with existing programs. This means that a large portion of Python code is written as scripts and command-line interfaces (CLI).

Building these command-line interfaces and tools is extremely powerful because it makes it possible to automate almost anything. As a result, CLIs can become quite complex over time—

It usually starts with a very simple script that runs a bit of Python code to do one specific thing. For example, access a web API and print the output to the console:

# print_user_agent.py
import requests

json = requests.get('http://httpbin.org/user-agent').json()
print(json['user-agent'])

You can simply run this using python print_user_agent.py and it will print out the name of the user agent used to make the API call.

As I said, a very simple script 😉

But what are your options when such a Python command-line script grows and becomes more complex?

That’s what we’ll be looking at throughout this tutorial. You’ll learn about the basics of building a CLI in Python and how click makes it a much better experience.

We’ll use that knowledge and go step-by-step from a simple script to a CLI with command-line arguments, options and useful usage instructions. All of this using the power of a framework called click.

At the end of this tutorial, you’ll know:

• Why click is a better alternative to argparse and optparse
• How to create a simple CLI with it
• How to add mandatory command-line arguments to your scripts
• How to parse command-line flags and options; and
• How you can make your command-line apps more user friendly by adding help (usage) text

And you’ll see how to achieve all of that with a minimal amount of boilerplate, too.

By the way, all of the code examples in this tutorial use Python 3.6. They might not work with earlier versions of Python, but if you run into any trouble leave a comment below and we’ll get it sorted out together.

Let’s get started!

Why should you write Python command-line scripts and tools?

The code snippet above is just an example and not very useful in real life. The scripts that I’ve written throughout my career as a Python developer are a lot more complex. They usually help build, test and deploy applications and make the process repeatable.

You might have your own experiences and know that this can be a large part of our daily work: Some scripts remain within the project they are built for. Others become useful to other teams or projects. They might even be extended with additional features.

In these cases, it becomes important to make the scripts more flexible and configurable using command-line parameters. It makes it possible to provide server names, credentials or any other piece of information to the script.

This is where Python modules like optparse and argparse come in and make your life a lot easier. But before we take a closer look at those, let’s get our terminology straight.

Basics of a command-line interface

A command-line interface (CLI) starts with the name of the executable. You type it’s name in the console and you access the main entry point of the script, such as pip.

Depending on the complexity of the CLI, you usually have parameters that you can pass to the script which can either be:

1. An argument, which is a mandatory parameter that’s passed to the script. If you don’t provide it, the CLI will return an error. For example, click is the argument in this command: pip install click.

2. Or it can be an option, which is an optional (🤯) parameter combining a name and a value portion such as --cache-dir ./my-cache. You tell the CLI that the value ./my-cache should be uses as the cache directory.

3. One special options is the flag which enables or disables a certain behaviour. The most common is probably --help. You only specify the name and the CLI interprets the value internally.

With more complex CLIs such as pip or the Heroku Toolbelt, you’ll get access to a collection of features that are all grouped under the main entry point. They are usually referred to as commands or sub-commands.

You’ve probably already used a CLI when you installed a Python package using pip install <PACKAGE NAME>. The command install tells the CLI that you’d like to access the feature to install a package and gives you access to parameters that are specific to this feature.

Command-line frameworks available in the Python 3.x standard library

Adding commands and parameters to your scripts is extremely powerful but the parsing of the command-line isn’t as straight forward as you would think. Instead of starting to write your own, you should use one of Python’s many packages that have solved this problem already.

The two most well-known packages are optparse and argparse. They are part of the Python standard library following the “batteries included” principle.

They mostly provide the same functionality and work very similar. The biggest difference is that optparse is deprecated since Python 3.2 and argparse is considered the standard for implementing CLIs in Python.

You can find more details on both of them in the Python documentation but to give you an idea what an argparse script looks like, here’s an example:

import argparse

parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('integers', metavar='N', type=int, nargs='+',
                    help='an integer for the accumulator')
parser.add_argument('--sum', dest='accumulate', action='store_const',
                    const=sum, default=max,
                    help='sum the integers (default: find the max)')

args = parser.parse_args()
print(args.accumulate(args.integers))

click vs argparse: A better alternative?

You’re probably looking at the code example above, thinking “what do any of these things mean?” And that’s exactly one of the problems I have with argparse: it’s unintuitive and hard to read.

That’s why I fell in love with click.

Click is solving the same problem as optparse and argparse but uses a slightly different approach. It uses the concept of decorators. This requires commands to be functions that can be wrapped using decorators.

Dan wrote a great introduction to decorators if this is the first time you hear the term or would like a quick refresher.

The author of click, Armin Ronacher, describes in a lot of detail why he wrote the framework. You can read the section “Why Click?” in the documentation and I encourage you to take a look.

The main reason why I use click is that you can easily build a feature-rich CLI with a small amount of code. And the code is easy to read even when your CLI grows and becomes more complex.

Building a simple Python command-line interface with click

I’ve talked enough about CLIs and frameworks. Let’s take a look at what it means to build a simple CLI with click. Similar to the first example in this tutorial, we can create a simple click-based CLI that prints to the console. It doesn’t take much effort:

# cli.py
import click

@click.command()
def main():
    print("I'm a beautiful CLI ✨")

if __name__ == "__main__":
    main()

First of all, let’s not worry about the last two lines for now. This is just Python’s (slightly unintuitive) way to run the main function when the file is executed as a script.

As you can see, all we have to do, is create a function and add the @click.command() decorator to it. This turns it into a click command which is the main entry point for our script. You can now run it on the command-line and you’ll see something like this:

$ python cli.py
I'm a beautiful CLI ✨

The beauty about click is, that we get some additional features for free. We didn’t implement any help functionality but you add the --help option and you’ll see a basic help page printed to the command-line:

$ python cli.py --help
Usage: cli.py [OPTIONS]

Options:
  --help  Show this message and exit.

A more realistic Python CLI example with click

Now that you know how click makes it easy to build a simple CLI, we are going to take a look at a slightly more realistic example. We’ll be building a program that allows us to interact with a Web API. Everyone uses them these days and they give us access to some cool data.

The API that we’ll look at for the rest of this tutorial is the OpenWeatherMap API. It provides the current weather as well as a five day forecast for a specific location. We’ll start with their sample API returning the current weather for a location.

I like to experiment with an API before I start writing code to understand better how it works. One tool that I think you should know about is HTTPie which we can use to call the sample API and see the result that it returns. You can even try their online terminal to run it without installation.

Let’s look at what happens when we call the API with London as the location:

$ http --body GET http://samples.openweathermap.org/data/2.5/weather \
  q==London \
  appid==b1b15e88fa797225412429c1c50c122a1
{
    "base": "stations",
    "clouds": {
        "all": 90
    },
    "cod": 200,
    "coord": {
        "lat": 51.51,
        "lon": -0.13
    },
    "dt": 1485789600,
    "id": 2643743,
    "main": {
        "humidity": 81,
        "pressure": 1012,
        "temp": 280.32,
        "temp_max": 281.15,
        "temp_min": 279.15
    },
    "name": "London",
    "sys": {
        "country": "GB",
        "id": 5091,
        "message": 0.0103,
        "sunrise": 1485762037,
        "sunset": 1485794875,
        "type": 1
    },
    "visibility": 10000,
    "weather": [
        {
            "description": "light intensity drizzle",
            "icon": "09d",
            "id": 300,
            "main": "Drizzle"
        }
    ],
    "wind": {
        "deg": 80,
        "speed": 4.1
    }
}

In case you’re looking at the screen with a face like this 😱 because the above example contains an API key, don’t worry that’s the sample API key they provide.

The more important observation from the above example is that we send two query parameters (denoted by == when using HTTPie) to get the current weather:

• q is our location name; and
• appid is our API key.

This allows us to create a simple implementation using Python and the Requests library (we’ll ignore error handling and failed requests for the purpose of simplicity.)

import requests

SAMPLE_API_KEY = 'b1b15e88fa797225412429c1c50c122a1'

def current_weather(location, api_key=SAMPLE_API_KEY):
    url = 'http://samples.openweathermap.org/data/2.5/weather'

    query_params = {
        'q': location,
        'appid': api_key,
    }

    response = requests.get(url, params=query_params)

    return response.json()['weather'][0]['description']

This function makes a simple request to the weather API using the two query parameters. It takes a mandatory argument location which is assumed to be a string. We can also provide an API key by passing api_key in the function call. It is optional and uses the sample key as a default.

And here’s our current weather for London form the Python REPL:

>>> current_weather('London')
'light intensity drizzle'  # not surprising 😉

# Nice:
$ python cli.py London

# Even better:
$ weather London

Parsing a mandatory parameter with click

The simple current_weather function allows us to build our CLI with a custom location provided by the user. I would like it to work similar to this:

$ python cli.py London
The weather in London right now: light intensity drizzle.

You probably guessed it already, the location in this call is what I introduced as an argument earlier. That’s because it is a mandatory parameter for our weather CLI.

How do we implement that in Click? It’s pretty straight forward, we use a decorator called argument. Who would’ve thought?

Let’s take the simple example from earlier and modify it slightly by defining the argument location.

@click.command()
@click.argument('location')
def main(location):
    weather = current_weather(location)
    print(f"The weather in {location} right now: {weather}.")

You can see that all we have to do is add an additional decorator to our main function and give it a name. Click uses that name as the argument name passed into the wrapped function.

In our case, the value for the command-line argument location will be passed to the main function as the argument location. Makes sense, right?

You can also use dashes (-) in your names such as api-key which Click will turned into snake case for the argument name in the function, e.g. main(api_key).

The implementation of main simply uses our current_weather function to get the weather for the location provided by the caller of our CLI. And then we use a simple print statement to output the weather information 🤩

Done!

And if that print statement looks weird to you, that’s because it is a shiny new way of formatting strings in Python 3.6+ called f-string formatting. You should check out the 4 major ways to format strings to learn more.

Parsing optional parameters with click

You’ve probably figured out a tiny flaw with the sample API that we’ve used above, you’re a smart 🍪

Yes, it’s a static endpoint always returning the weather for London from January 2017. So let’s use the actual API with a real API key. You can sign up for a free account to follow along.

The first thing we’ll need to change is the URL endpoint for the current weather. We can do that by replacing the url in the current_weather function to the endpoint in the OpenWeatherMap documentation:

def current_weather(location, api_key=SAMPLE_API_KEY):
    url = 'https://api.openweathermap.org/data/2.5/weather'

    # everything else stays the same
    ...

The change we just made will now break our CLI because the default API key is not valid for the real API. The API will return a 401 UNAUTHORIZED HTTP status code. Don’t believe me? Here’s the proof:

$ http GET https://api.openweathermap.org/data/2.5/weather q==London appid==b1b15e88fa797225412429c1c50c122a1
HTTP/1.1 401 Unauthorized
{
    "cod": 401,
    "message": "Invalid API key. Please see http://openweathermap.org/faq#error401 for more info."
}

So let’s add a new parameter to our CLI that allows us to specify the API key. But first, we have to decide if this should be an argument or an option. I say we make it an option because adding a named parameter like --api-key makes it more explicit and self-documenting.

Here’s how I think the user should run it:

$ python cli.py --api-key <your-api-key> London
The weather in London right now: light intensity drizzle.

That’s nice and easy. So let’s see how we can add it to our existing click command.

@click.command()
@click.argument('location')
@click.option('--api-key', '-a')
def main(location, api_key):
    weather = current_weather(location, api_key)
    print(f"The weather in {location} right now: {weather}.")

Once again, we are adding a decorator to our main function. This time, we use the very intuitively named @click.option and add in the name for our option including the leading double dashes (--). As you can see, we can also provide a shortcut option with a single dash (-) to save the user some typing.

I mentioned before that click creates the argument passed to the main function from the long version of the name. In case of an option, it strips the leading dashes and turns them into snake case. --api-key becomes api_key.

Last thing we have to do to make this work is passing the API key through to our current_weather function. Boom 👊🏼

We’ve made it possible for our CLI user to use their own key and check out any location:

$ python cli.py --api-key <your-api-key> Canmore
The weather in Canmore right now: broken clouds.

And looking out my window, I can confirm that’s true 😇

Adding auto-generated usage instructions to your Python command-line tool

You can pat yourself on the back, you’ve built a great little CLI with a minimal amount of boilerplate code. But before you take a break and enjoy a beverage of your choice. Let’s make sure a new user can learn how to run our little CLI…by adding some documentation (don’t run, it’ll be super easy.)

First let’s check and see what the --help flag will display after all the changes that we’ve made. As you can see, it’s not bad for no effort at all:

$ python cli.py --help
Usage: cli.py [OPTIONS] LOCATION

Options:
  -a, --api-key TEXT
  --help              Show this message and exit.

The first thing that we want to fix is the missing description for our API key option. All we have to do is provide a help text to the @click.option decorator:

@click.command()
@click.argument('location')
@click.option(
    '--api-key', '-a',
    help='your API key for the OpenWeatherMap API',
)
def main(location, api_key):
    ...

The second and final change we’ll make is adding documentation for the overall click command. And the easiest and most Pythonic way is adding a docstring to our main function. Yes, we should do that anyways, so this isn’t even extra work:

...
def main(location, api_key):
    """
    A little weather tool that shows you the current weather in a LOCATION of
    your choice. Provide the city name and optionally a two-digit country code.
    Here are two examples:

    1. London,UK

    2. Canmore

    You need a valid API key from OpenWeatherMap for the tool to work. You can
    sign up for a free account at https://openweathermap.org/appid.
    """
    ...

Putting it all together, we get some really nice output for our weather tool.

$ python cli.py --help
Usage: cli.py [OPTIONS] LOCATION

  A little weather tool that shows you the current weather in a LOCATION of
  your choice. Provide the city name and optionally a two-digit country
  code. Here are two examples:

  1. London,UK

  2. Canmore

  You need a valid API key from OpenWeatherMap for the tool to work. You can
  sign up for a free account at https://openweathermap.org/appid.

Options:
  -a, --api-key TEXT  your API key for the OpenWeatherMap API
  --help              Show this message and exit.

I hope at this point you feel like I felt when I first discovered click: 🤯

Python CLIs with click: Summary & Recap

Alright, we’ve covered a ton of ground in this tutorial. Now it’s time for you to feel proud of yourself. Here’s what you’ve learned:

• Why click is a better alternative to argparse and optparse
• How to create a simple CLI with it
• How to add mandatory command-line arguments to your scripts
• How to parse command-line flags and options; and
• How you can make your command-line apps more user friendly by adding help (usage) text

And all of that with a minimal amount of boilerplate! The full code example below illustrates that. Feel free to use it for your own experiments 😎

import click
import requests

SAMPLE_API_KEY = 'b1b15e88fa797225412429c1c50c122a1'


def current_weather(location, api_key=SAMPLE_API_KEY):
    url = 'https://api.openweathermap.org/data/2.5/weather'

    query_params = {
        'q': location,
        'appid': api_key,
    }

    response = requests.get(url, params=query_params)

    return response.json()['weather'][0]['description']


@click.command()
@click.argument('location')
@click.option(
    '--api-key', '-a',
    help='your API key for the OpenWeatherMap API',
)
def main(location, api_key):
    """
    A little weather tool that shows you the current weather in a LOCATION of
    your choice. Provide the city name and optionally a two-digit country code.
    Here are two examples:
    1. London,UK
    2. Canmore
    You need a valid API key from OpenWeatherMap for the tool to work. You can
    sign up for a free account at https://openweathermap.org/appid.
    """
    weather = current_weather(location, api_key)
    print(f"The weather in {location} right now: {weather}.")


if __name__ == "__main__":
    main()

If this has inspired you, you should check out the official click documentation for more features. You can also check out my introduction talk to click at PyCon US 2016. Or keep an eye out for my follow up tutorial where you’ll learn how to add some more advanced features to our weather CLI.

Happy CLI coding!